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What we examined 


Financial Management and Control 
and Risk Management 


Main Points 


Financial control and risk management are central to managing any 
organization effectively. To the extent that they are done well, they can 
contribute to an organization’s ability to safeguard its assets, use its 
resources economically and efficiently, and produce accurate and 
reliable financial information. Reliable information is key to 
developing a complete and accurate picture of financial performance. 


In our April 2003 Report we observed that federal departments we had 
selected for audit were in the early stages of implementing integrated 
risk management and that much remained to be done. For example, 
departments needed to complete their analysis of risk, including 
assessing their tolerance for risk. We also noted that departments 
needed more guidance from the Treasury Board of Canada Secretariat 
on the steps involved in integrated risk management. 


Our May 2006 Report noted that departments selected for that audit 
had made some progress in resolving financial control weaknesses and 
improving their financial management competencies. However, we 
noted that unless they also improved their financial controls, the 
quality of information used for many key decisions would be at risk. 
We also noted that most financial information used for budgeting and 
decision making was based on the cash method of accounting, which 
provided a less accurate and complete picture of a department’s 
financial situation than the accrual method. 


In this follow-up audit we examined government's progress in acting 
on commitments made in response to our previous concerns. Our 
audit included the Treasury Board of Canada Secretariat (the 
Secretariat) which includes the Office of the Comptroller General, and 
seven large federal departments, namely: Agriculture and Agri-Food 
Canada, Department of Finance Canada, Foreign Affairs and 
International Trade Canada, Human Resources and Skills 
Development Canada, Indian and Northern Affairs Canada, Transport 
Canada, and Veterans Affairs Canada. 


Audit work for this chapter was substantially completed 
on 30 September 2010. 
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Why it’s important © Canadians expect the government to be well managed and to be 
accountable for the safeguarding of public assets and the 
stewardship—efficient, effective, and economical use—of public funds. 
Effective financial controls and risk management are vital to reduce 
the risk that the government may not achieve these objectives. 


What we found + Overall, the Treasury Board of Canada Secretariat and departments 
have made satisfactory progress in acting on their previous 
commitments, in particular, with respect to developing policies, 
frameworks, and relevant guidance on internal controls, and 
implementing risk management measures. Departments have still 
not fully assessed their internal control systems to identify and 
address weaknesses; and, according to their action plans, this work 
will take years to complete. In addition, the Office of the 
Comptroller General (OCG) has yet to establish processes for 
monitoring the completion of these assessments or for assessing 
actions taken by departments to address significant internal control 
issues. There is also a long-standing issue that has not been 
sufficiently addressed—namely, accrual-based budgeting and 
appropriations. 


- The Treasury Board of Canada Secretariat, through the Office of the 
Comptroller General, has demonstrated leadership by developing a 
new and strengthened financial management policy suite, clarifying 
roles and responsibilities, and supporting deputy heads in their role 
as accounting officers as set out in the Financial Administration Act. 
Most of the selected departments told us they found the Secretariat's 
initiatives and approach to be helpful. However, there is more work 
to be done to improve the Secretariat’s monitoring of the 
implementation of these policies so that it can appropriately assess 
departments’ performance in achieving the policy goals. 


Each of the selected departments has developed a corporate risk 
profile that summarizes an assessment of the department's key risks, 
with processes in place to update these profiles regularly. The 
integration of risk into planning, reporting, and decision making has 
improved. 


Financial human resource capacity has improved since our last 
report. The Office of the Comptroller General has put in place 
several measures to enhance capacity, including guidelines on the 
qualifications of chief financial officers, mandatory training courses 
for financial staff, promotion of professional accounting designations 
for mid-level financial officers and financial managers, staffing 
initiatives, and forums and exchanges on best practices. 
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Departments are working to maintain and strengthen their 
workforce by recruiting, training, and developing the appropriate 
number of financial officers and managers with the required skills, 
competencies, and experience. Departments are at various stages in 
the process of putting in place strategies to address the anticipated 
turnover of senior financial executives. 


The government has not completed its evaluation of accrual-based 
budgeting and accrual-based appropriations and has not yet decided 
if it intends to fully adopt either or both of these approaches. It has 
taken what it considers the needed action related to accrual-based 
budgeting. However, it is our view that these actions are not 
sufficient or appropriate to attain accrual-based budgeting at the 
departmental level. At this point no decision on whether to adopt 
accrual-based appropriations has been made. 


The departments and the Secretariat have responded. The 
departments and the Secretariat agree with all of the recommendations. 
Their detailed responses follow the recommendations throughout 

the chapter. 
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FINANCIAL MANAGEMENT AND CONTROL AND RISK MANAGEMENT 


Introduction 


1.1 Each year, the government manages billions of dollars in taxpayers’ 
money to deliver services and programs to Canadians. The Public 
Accounts of Canada reported that during the 2009-10 fiscal year, the 
Government of Canada had $219 billion in revenue and $274 billion in 
expenses. In addition, the government’s financial position at that time 


included assets of $364 billion and liabilities of $883 billion. 


1.2 Canadians expect the government to be well managed and to 
handle taxpayers’ money prudently and economically. To do this, the 
government must exercise sound financial management and control 
and risk management within the requirements of the Financial 
Administration Act and regulations. It must also follow relevant 
Treasury Board policies, directives, and standards, and be guided by 
principles of value for money, accountability, transparency, and risk 
management. This means that systems, practices, and resources need 
to be in place to ensure that 


* public funds are managed prudently and honestly; 
* assets are safeguarded; 


* resources are used effectively, efficiently, and economically to 


achieve government objectives; 
* accountabilities for financial management are clearly established; 
* financial risks are being mitigated by effective internal controls; and 


¢ the Canadian public and parliamentarians are provided with 
pertinent and reliable information on the use of public funds. 


Significant deficiencies in these areas of financial management and 
control and in risk management can have detrimental effects on the 
level of service that Canadians receive and the government's credibility. 


13 To be well managed an organization must identify, assess, 

and respond to risks that could affect the fulfillment of its objectives. 
Integrated risk management involves managing risk across an 
organization in a coordinated and systematic manner. This assists an 
organization in better understanding the risks it faces—for example, 
financial, environmental, strategic, and operational risks. It also assists 
an organization in making informed decisions about how to mitigate 
threats that could have an impact on its ability to meet its objectives, 
and how to take advantage of the opportunities that uncertainty 


presents. 
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Accrual-based financial information— 
Information compiled using a method of 
accounting that records an entity's transactions 
and other economic events when they occur, 
rather than when the entity receives or pays 
cash or its equivalent (cash-based accounting). 


Appropriations—An authority from Parliament 
to incur a maximum dollar amount for a 
specified activity during a fiscal year, with 
legislated consequences for exceeding the 
maximum established by the appropriation. 


14 Good financial management requires the provision of pertinent 
and reliable information for planning and allocating funds. It is our 
view that the use of accrual-based financial information for 
budgeting and appropriation of funds would allow departments to 
better manage and control operating and capital spending and improve 
transparency and accountability. It would also help legislators hold the 
government accountable for its stewardship (effective, efficient, and 
economical use) of public assets, the full costs of programs, and its 
ability to meet short-term and long-term financial obligations. 


1.5 The capacity of financial human resources also plays a key role in 
determining how well the government can deliver results that meet the 
expectations of Canadians. In a climate of fiscal pressure, it is important 
for departments to continue to maintain and build a strong workforce 
for the future. This can be accomplished by recruiting, training, and 
developing the appropriate number of financial officers and managers 
with the required qualifications, skills, competencies, and experience. 


1.6 Exhibit 1.1 summarizes key roles and responsibilities for 
implementing financial management and control and risk management. 


Exhibit 1.1 Summary of key roles and responsibilities for implementing financial management and control and risk management 


Deputy head 


The Chief Financial Officer 
Senior departmental managers 


The Treasury Board of Canada 
Secretariat 


As the “accounting officer,” has overall responsibility for financial management and 
control and risk management within a department. This includes accountability for 


* allocating resources to deliver departmental programs in compliance with 
government policies and procedures; 


appointing a qualified Chief Financial Officer; 


maintaining effective systems of internal controls and signing accounts in a manner 
that accurately reflects the financial position of the department; and 


exercising any other duties prescribed by law or regulations relating to the 
administration of their department (in practice, accounting officers are also 
accountable for parliamentary appropriations received). 


Provides support to the deputy head as the senior departmental executive, strategic 
adviser, and key steward for departmental financial management. ; 


Are responsible for financial and risk management for all the activities falling within his 
or her areas of responsibility. 


¢ Serves as a centre of expertise to support integrated risk management and sharing 
of good practices. 


* Provides guidance and advice on risk management for departments and agencies. 


¢ Annually examines and evaluates the effectiveness of integrated risk management 
as well as financial management and control within government departments and 
agencies and reports on progress. 
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Exhibit 1.1 Summary of key roles and responsibilities for implementing financial management and control and risk management (continued) 


The Office of the Comptroller General of 
Canada within the Secretariat 


Provides government-wide functional direction and guidance to departments and 
agencies on financial management and control, and on stewardship of public resources 
and financial management information systems. This includes 


* interpreting financial management policies, directives, and standards for the 
Goverament of Canada; 


* assessing departmental financial management performance periodically; 


* supporting the development of sustainable financial management capacity across 
the government; and 


* monitoring compliance with policy, reporting on deficiencies, and ensuring that 
departments take timely corrective action. 


Sources: Treasury Board Policy on Financial Management Governance, Treasury Board Policy on Internal Control, Framework for the Management of Risk, and 
Treasury Board Guidelines on the Expected Qualifications for Chief Financial Officers. 


What we found in our 2003 and 2006 audits 


17. This chapter follows up on selected recommendations and 
relevant observations, and on the government's related commitments, 
from our 2003 April Report of the Auditor General of Canada, 
Chapter |, Integrated Risk Management, and our 2006 May Status 
Report of the Auditor General of Canada, Chapter 1, Managing 
Government: Financial Information. 


1.8 In 2003, we reported that the Treasury Board of Canada 
Secretariat and the departments that we had selected for audit were at 
an early stage of implementing integrated risk management. We noted 
that the Secretariat needed to provide more guidance on how to 
integrate risk management and that departments needed to 


* complete risk profiles, 
* better define risk tolerances, 


* periodically review profiles to ensure that risks remained 
relevant, and 


* integrate risk information into strategic decision making. 


19 In 2006, we followed up on long-standing issues related to 
financial management and controls and reported that the Office 

of the Comptroller General had not done enough to enhance 
financial management and control government-wide and to promote 
accrual-based budgeting and appropriations. We also reported that 
less than 50 percent of the selected federal organizations had made 
satisfactory progress in addressing previously reported weaknesses in 
financial controls. 
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Events since 2006 


1.10 In December 2006, Parliament passed the Federal Accountability 
Act. Shortly thereafter, the government presented the Federal 
Accountability Action Plan to increase government-wide 
accountability, transparency, and oversight in departments and 
agencies. In this plan, the government committed to reviewing and 
streamlining financial management policies; replacing transaction-level 
requirements and rules with basic principles of accountability and 
transparency, where possible; and revoking policies and regulations that 
did not promote accountability and good management. To meet these 
objectives, the Secretariat continued its policy renewal initiative. 


1.11 Amendments to the Financial Administration Act made by the 
Federal Accountability Act, named the deputy head of a department 

or an agency as the “accounting officer.” These amendments clearly 
established the deputy head’s accountability before parliamentary 
committees for measures taken to ensure the department’s compliance 
with government policies and procedures. They also set out the deputy 
head's responsibilities for maintaining effective internal controls. 


112 In support of the Federal Accountability Action Plan, the 
Secretariat reviewed the Treasury Board’s management policies to 
clarify management’s roles and responsibilities and to ensure a risk- 
based approach to streamlining rules and processes. This was also 
intended to clarify the objectives of Treasury Board policies and the 
results to be achieved by them and to clarify the Secretariat’s oversight 
role in holding departments accountable for management performance, 
including maintaining effective internal controls. Exhibit 1.2 
summarizes key financial management policies. 


Focus of the audit 


1.13 In this chapter, we reviewed the progress that the Secretariat 

and seven selected departments, listed in About the Audit, had made 
toward commitments related to selected recommendations and relevant 
observations from our 2003 and 2006 audits. We looked at whether they 
had taken satisfactory action to enhance controls for financial reporting 
and to improve integrated risk management. We also looked at steps that 
the Secretariat had taken toward implementing accrual-based budgeting 
and appropriations. Although our previous audits did not make specific 
recommendations on the capacity of financial management human 
resources, we considered this important aspect of financial management 
and control in this follow-up report. We did not assess internal controls 
within the selected departments or the risks identified in their Corporate 
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Risk Profile. Rather, we focused on whether the departments were 
assessing their internal controls and practices, and on their approach 
to risk management. 


1.14 More details about the audit objectives, scope, approach, 
and criteria are in About the Audit at the end of this chapter. 


Exhibit 1.2 Summary of key financial management policies 


Policy o on Financial Management Effective date: 1 April 2009 
Governance 


Objective: To strengthen public sector financial management and its leadership thereby 
contributing to appropriate stewardship of public resources, effective decision-making, 
and efficient policy and program delivery. 


Requirements: The policy specifies core financial management responsibilities 
pertaining to deputy heads, chief financial officers, senior departmental managers, 
departmental audit committees and the Comptroller General. This includes that the - 
deputy head must appoint a Chief Financial Officer. 


a 


Policy on Internal Control Effective date: 1 April 2009 


Implementation was to be phased in over three years, based on risks and departmental 
readiness as follows: 


¢ 1 April 2009—effective for the 22 large departments that completed a readiness 
assessment to support control-based audits of financial statements 


¢ 1 April 2010—effective for departments and agencies that currently have audits of 
their financial statements 


¢ 1 April 201 1—effective for all other departments and agencies 


Objective: To adequately manage risks relating to the stewardship of public resources 
through effective internal controls, including internal controls for financial reporting. 


Requirements: Deputy heads are responsible for 


* establishing, maintaining, monitoring and reviewing an effective system of internal 
controls to mitigate risk with respect to the effectiveness and efficiency of programs, 
operation and resource management, the reliability of financial reporting and 
compliance with legislation, regulations, policies and delegated authorities; 


* conducting an annual risk-based assessment of the system of internal control over 
financial reporting; 


¢ establishing an action plan to address significant issues found in the assessment 
and ensuring appropriate and timely action is taken to address issues; and 


* publishing a summary of the assessment results and actions taken in response. 


The Office of the Comptroller General within the Treasury Board of Canada Secretariat 
is responsible for 


* overseeing the effectiveness of the system of internal control; and 


* monitoring that appropriate and timely action is taken to address significant issues 
over financial management and over financial reporting. 
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Exhibit 1.2 Summary of key financial management policies (continued) 


Effective date: 1 June 2010, except for the requirement for quarterly reporting, which 
’ Management, Information and Reporting becomes effective 1 April 2011 


Objective: To ensure that the management of public funds is supported by effective 
planning, budgeting, and reporting, which are based on reliable information and the 
sound analysis of that information. 


Requirements: Deputy heads are responsible for ensuring 


* approval, timely allocation, and effective oversight of the departments’ financial 
plans, budgets, and related allocation of resources with appropriate involvement of 
the Chief Financial Officer; 


* preparation of a complete set of departmental financial statements in accordance 
with Treasury Board standards; 


* preparation of quarterly reports to be made public within 60 days of the reporting 
period; and 


* implementation of measures for the department to sustain a control-based audit of 
its annual financial statements in whole or in part. 


The Treasury Board of Canada Secretariat is responsible for requesting audits of 
departments’ financial statements in part or in whole and monitoring compliance with 


Policy. 
Policy on Stewardship of Financial Effective date: 1 January 2010 (replaces the policy on Financial Systems and Controls 
Management Systems dated 1996) 


Objective: To ensure that financial management systems deliver more accurate, 
reliable, accessible, and timely financial information. 


Requirements: Deputy heads are responsible for ensuring that 


* financial management systems (FMS) meet operational requirements and are 
aligned with government-wide strategic direction for FMS; 


* configurations are standardized; and, 


* common government-wide financial management business processes are used and 
common data requirements are in place. 


The Treasury Board of Canada Secretariat is responsible for 


* ensuring that sound financial management governance structures are in place for 
each cluster group; 


* supporting Treasury Board in issuing appropriate policy instruments 


* monitoring compliance with Policy 


Sources: Treasury Board Policy on Financial Management Governance, Treasury Board Policy on Internal Control, Treasury Board Policy on Financial Resource 
Management, Information and Reporting, and Treasury Board Policy on Stewardship of Financial Management Systems 
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Observations and Recommendations 


Financial controls = 1.15 In response to the recommendations on internal controls for 
financial reporting made in our 2006 May Status Report, Chapter 1, 
the Treasury Board of Canada Secretariat committed to improving and 
strengthening the policy environment related to financial controls. 
In 2005, as apart of the audited financial statement initiative, the 
Secretariat had requested that large departments undergo a readiness 
assessment, which included the review of the control environment 
in which each department operated. At that time, the government’s 
objective was to improve the integrity of financial information both 
for decision making and accountability. The purpose of the readiness 
assessment was to determine whether departments were ready to 
support a controls-based audit of such departmental financial 
statements. The Office of the Comptroller General within the 
Secretariat was to monitor results of the assessment and work with 
departments to ensure they took corrective action where necessary. 


1.16 At the time of our 2006 audit, some departments were in the 
process of having this assessment done. In 2008, when all the readiness 
assessments were completed, none of the selected departments could 
demonstrate that they had sufficient controls in place to support a 
controls-based audit of financial statements. Most of these departments 
began work to improve their systems of internal control at that time. 
Subsequently, the departments’ focus shifted from audited financial 
statements to the mandatory annual public disclosure of their risk-based 
assessment of controls over financial reporting and of their planned 
improvements. These requirements are set out in the 2009 Treasury 
Board Policy on Internal Control. 


1.17 We examined the steps taken by the Secretariat to improve the 
policy environment for financial controls, including whether it had 
issued its policy instruments and whether the Office of the Comptroller 
General was supporting departments in implementing the policy 
requirements and monitoring departments’ compliance with these 
requirements. We also examined whether the selected departments 
had assessed their internal controls for financial reporting and 
addressed gaps and weaknesses identified in order to work toward 
implementing the new policies. 
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The Secretariat has developed policies, directives, and standards for financial 
controls and is supporting departments in applying them 


1.18 Development of policies, directives, and standards. We found 
that the Secretariat has developed new and stronger policy 
instruments to improve the financial and risk management capacities 
of departments at all levels. The Policy on Internal Control, which 
came into effect on 1 April 2009 for the 22 largest departments 
(including the 7 departments selected for this audit), is a critical 
element of the government’s approach. This policy defines 
responsibilities for establishing, maintaining, communicating, and 
monitoring internal controls for managing public resources. It is 
designed to manage risk and support deputy heads, in fulfilling their 
responsibilities for providing reasonable assurance that their financial 
reporting controls are well maintained and that resources are well 
managed. The policy emphasizes having effective financial reporting 
controls to mitigate significant financial risks. It requires departments 
to perform an annual risk-based assessment of internal financial 
reporting controls and disclose the results of such assessments to the 
public, as well as their plans to address any changes needed. The intent 
of this policy is to increase public confidence and trust through 
strengthened accountability and transparency. 


119 Support and monitoring. We found that the Office of the 
Comptroller General (OCG) is providing ongoing support to 
departments as they implement the policies and related directives and 
standards and has some mechanisms in place to monitor government- 
wide progress. The OCQ’s related actions include 


* continued use of “policy centres” to provide guidance and support 
to departments and agencies, as they implement Treasury Board 
policies on financial management, and to facilitate the sharing of 
information and best practices across departments; 


* development of guidelines and tools, in consultation with 
departments, for implementing policy requirements (for example, 
the new Guideline on Chief Financial Officer Qualifications and 
Guideline on Internal Control Over Financial Reporting); 


* some improvements to the monitoring of financial management 
across the government, through ongoing modifications to its 
Management Accountability Framework (an assessment tool used 
to evaluate departmental performance against the Secretariat’s 
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expectations for good management, including financial 
management); and 


* audits of various government departments to assess performance 
on implementation of policies. 


1.20 Officials from the selected departments informed us that they are 
pleased with the support that the OCG has provided on this initiative 
and are particularly satisfied with the consultative approach taken. 


1.21 The OCG reviews departments’ annual disclosure of the results 
of work done to date to assess internal controls for financial reporting. 
However, at the time of the audit, the OCG had not yet established 
processes for monitoring the completion of these assessments or for 
assessing actions taken by departments to address significant issues 
identified as required by the Policy on Internal Control. Moreover, 
there was no plan in place to determine when departments ought to 
identify or remedy any major weaknesses in their financial reporting 
controls. In addition, there were no plans to assess and monitor the 
progress in addressing those weaknesses (Exhibit 1.3). In our view, the 
OCG needs to increase its efforts in monitoring policy requirements. 


Exhibit 1.3 Progress in addressing our recommendation on providing guidance to departments 
on internal financial controls 


3 n ae dh is eae . 
we ey => a NGI aU 


The Office of the Comptroller General should 


¢ work closely with internal auditors and senior financial officials 
in departments and agencies to develop a strategy for 
strengthening internal financial controls, 


¢ develop an implementation plan that includes deadlines by 
which departments and agencies must identify and remedy any 
weaknesses in their internal financial control systems and that 
provides for assessing and monitoring the progress in 
addressing those weaknesses, and 


Unsatisfactory 


regularly monitor the actions of departments and agencies to 
assess their progress against this plan. 


(Recommendation 1.37 of the 2006 May Status Report of the 
Auditor General of Canada, Chapter 1, Managing Government: 
Financial Information) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 
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The selected departments are implementing the Policy on Internal Control but have 
not fully assessed their internal controls for financial reporting 


1.22 We found that the seven selected departments have made some 
progress toward completing their annual risk-based assessments of 
financial reporting controls, as required by the Policy on Internal 
Control (Exhibit 1.4). These assessments and the related actions taken 
to address identified weaknesses are essential to having internal 
controls that are properly designed and implemented to address 
financial risks of the entity. Departments that are more advanced 

in carrying out these assessments have identified weaknesses in the 
design of their key financial controls, such as gaps in supporting 
documentation or weaknesses in controls over IT management 
processes (for example, access and change management processes). 
These departments are implementing action plans to address the 
weaknesses in design of controls. We have not audited the internal 
controls and therefore do not conclude whether internal controls are 
effective. However, we do note that the selected departments have not 
yet done all the work required under the Policy on Internal Control to 
assess their financial reporting controls. 


1.23 In November 2010, we noted that the selected departments 
had included their progress and the expected date for completing the 


Exhibit 1.4 Status of departments’ assessments of financial reporting controls as of 30 September 2010 


Agriculture and Agri-Food Canada 2012-13 
Department of Finance Canada 


2010-11 
Foreign Affairs and International 


Human Resources and Skills 
Development Canada Not Started 2013-14 
Indian and Northern Affairs 

Not Started 2014-15 


Canada 


Transport Canada 
Veterans Affairs Canada 2010-11 


@ Substantially complete—Most required actions are completed 
w Moderately complete—Additional action is still required 


i) Partially complete—Considerable action is still required 
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assessment of their financial reporting controls in an appendix to their 
departmental financial statements, which are available to the public on 
the departments’ websites. 


1.24 In summary, although the selected departments are making 
progress, much work remains to be done. It is expected that it will take 
years for most of the departments to complete the first full assessment 
of the design and effectiveness of their financial reporting controls. 
Considering the importance of ensuring that internal controls are in 
place and operating effectively, the timelines presented in Exhibit 1.4 
may be lengthy. Furthermore, the departments have not fully 
implemented a program for the continuous monitoring of the 
effectiveness of their internal controls. 


1.25 Recommendation. The seven audited departments (Agriculture 
and Agri-Food Canada, Department of Finance Canada, Foreign 
Affairs and International Trade Canada, Human Resources and Skills 
Development Canada, Indian and Northern Affairs Canada, Transport 
Canada, and Veterans Affairs Canada), in consultation with the Office 
of the Comptroller General, should review the timelines set for 
completing their full risk-based assessment of the effectiveness of their 
financial reporting controls and for addressing identified gaps and 
weaknesses, to ensure that they are timely. The departments should 
also implement programs for continuous monitoring of these controls 
to confirm their ongoing effectiveness. 


The departments’ response. Agreed. The selected departments are in 
the process of carrying out risk-based action plans for completing 
assessments of financial reporting controls within the current 
timelines. The departments generally agree to review these timelines in 
consultation with the Office of the Comptroller General, as well as any 
timelines they have or will set-for addressing identified gaps and 
weaknesses in controls. Furthermore, the departments will continue to 
implement programs for ongoing monitoring of their financial controls. 


1.26 Recommendation. The Treasury Board of Canada Secretariat 
should continue to review the departments’ implementation of the 
Treasury Board Policy on Internal Controls to support the effectiveness 
of internal controls for financial reporting across government. This 
should include monitoring the departments’ established timelines for 
completing their risk-based assessments of these controls. The 
Secretariat should also monitor the departments’ actions to address 
significant issues identified as a result of their assessments, including 
monitoring that the actions are appropriate and timely. 
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Chapter 1 


The Secretariat’s response. Agreed. The Secretariat continues to 
acknowledge the importance of monitoring and strengthening internal 
financial controls across government. The foundational Policy on 
Internal Control, in place since 1 April 2009, is a risk-based policy 
that advocates effective internal control over financial reporting in 
departments. A three-year, phased-in approach has been taken to 

the implementation of the policy and includes a requirement for 
departments to publish the results and action plans arising from 
assessments of the effectiveness of internal controls over financial 
reporting. 


In implementing the Treasury Board’s Policy on Internal Control, the 
Office of the Comptroller General (OCG) supports departments and 
the government's financial management community, by reaffirming 
the policy expectations and requirements and by providing tools 

and guidance. The OCG also monitors departmental progress in 
implementing the policy in a variety of ways, including, but not limited 
to, reviews of the annual departmental disclosure on internal controls 
over financial reporting, required under the Policy on Internal Control; 
assessments under the Management Accountability Framework; 
results of audits and evaluations; and special audits or horizontal audits 
that may be commissioned by the Comptroller General of Canada. 


1.27 In response to our previous recommendations on integrated risk 
management, the Treasury Board of Canada Secretariat made a 
commitment to provide departments with ongoing support, guidance, 
and tools to raise awareness, build understanding, and support 
implementation of integrated risk management in government 
departments. It also made a commitment to encourage departments to 
implement the recommendations included in our previous report. 


1.28 We examined the Secretariat’s progress related to these 
commitments. Although individual departments made no 
commitments in response to our 2003 recommendations, we considered 
whether the selected departments had an integrated risk management 
function and an approach to integrating risk management into 
operations. We also assessed whether the selected departments had 
implemented key elements of corporate risk profiles and integrated risk 
management. We did not assess the completeness or appropriateness of 
the risks identified by the departments in the corporate risk profile. 
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The Secretariat’s guidance and support to departments has improved 


1.29 We found that the Secretariat has recently taken steps to 
improve guidance and support to departments for developing and 
implementing risk management within the departments. These steps 
included 


* re-establishing the Centre of Excellence on Risk Management 
in 2008 to provide ongoing guidance to departments; 


* issuing a new principle-based Framework for the Management 
of Risk in August 2010, which identifies the deputy head’s 
responsibility for risk management and the Secretariat’s 
responsibility to provide guidance, tools, and expertise to 
support departments; 


* finalizing guides and tools in support of the new framework, 
including the Guide to Corporate Risk Profiles, Guide to 
Integrated Risk Management, Guide to Risk Taxonomies, and 
Risk Management Capability Model, as well as drafting the Risk 
Management Guide for Deputy Heads; and 


* organizing forums and interdepartmental working groups for 
sharing best practices. 


It is too early to assess whether the recent guidance will be effective 
in assisting departments in their ongoing efforts to improve integrated 
risk management. 


The selected departments have corporate risk profiles and are integrating risk 
management into their operations 


1.30 We found that all of the selected departments have an integrated 
risk management function in place. They have secured the resources 
needed to carry out this function, which is geared to identifying, 
assessing, and mitigating corporate risks. We noted that the 
departments are working to continually improve their corporate risk 
profiles and to further enhance the integration of risk management 
within their organizations. 


1.31 Corporate risk profiles. We found that each of the selected 
departments has a corporate risk profile that outlines management’s 
assessment of the department’s environment and the key risks facing 
the department. Exhibit 1.5 is a summary of the departments’ progress 
on implementing the main elements of their corporate risk profiles. 
This exhibit illustrates that progress has been satisfactory for all of the 
selected departments. Nevertheless, department officials recognize 
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in financial management 


that integrated risk management is an ongoing challenge and will 
continue to enhance their risk management processes, including those 
related to the application of risk tolerances, as well as the training of 
staff and the communication of risk to department staff. 


1.32 Integration of risk management into operations. We found 
that most of the selected departments have integrated risk information 
into business planning, priority setting, decision making, and reporting 
(Exhibit 1.6). In addition, we found that all selected departments have 
made satisfactory progress in implementing our recommendations on 
integrated risk management (Exhibit 1.7). 


1.33. Under the Treasury Board Policy on Financial Management 
Governance, the Office of the Comptroller General (OCG) is to 
provide leadership and direction to strengthen human resource capacity 
in the area of financial management across government departments. 
Given the current economic climate and the importance of financial 
management, departments understandably are working hard to 
maintain and strengthen their workforce for the future by recruiting, 
training, and developing the appropriate number of financial officers 
and managers with the required skills, competencies, and experiences. 


Exhibit 1.5 Progress in implementing key elements of a corporate risk profile 


Agriculture and Agri-Food Canada 


Department of Finance Canada 


Foreign Affairs and International Trade 


Canada 


Human Resources and Skills 
Development Canada 


Indian and Northern Affairs Canada 


Transport Canada 


Veterans Affairs Canada 


Satisfactory Satisfactory 
Satisfactory Satisfactory 
Satisfactory Satisfactory 
Satisfactory Satisfactory 
Satisfactory Satisfactory 
Satisfactory Satisfactory 
Satisfactory Satisfactory 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, and the time that has elapsed since 
the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, and the time that has elapsed since 
the recommendation was made. 


Source: see Recommendation 1.65 of the 2003 April Report of the Auditor General of Canada, Chapter 1, Integrated Risk Management 
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Exhibit 1.6 Progress in implementing key elements of risk integration 


encuure ae Aeri- oad eee Sei ctaciery Satisfactory Satisfactory 


Department of Finance Canada Satisfactory Satisfactory , Satisfactory 


Foreign Affairs and International Trade 
Canada 


Human Resources and Skills ' j : 

Development Canada Satisfactory | Satisfactory Satisfactory 
- =e. = a SoS oak Se Ee ee 

Indian and Northern Affairs Canada Satisfactory Satisfactory Satisfactory 


Transport Canada Satisfactory Unsatisfactory Satisfactory 


Veterans Affairs Canada Satisfactory Satisfactory Satisfactory 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, and the time that has elapsed since 
the recommendation was made. 


Satisfactory ; Satisfactory Satisfactory 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, and the time that has elapsed since 
the recommendation was made. 


Source: see Recommendation 1.48 of the 2003 April Report of the Auditor General of Canada, Chapter 1, Integrated Risk Management 


Exhibit 1.7 Overall progress in addressing our recommendations on integrated risk management 


The Peon, se Sekrentit shane mide departments and agencies with iE I guidance 
and specific tools to assist them in developing the key capabilities needed to integrate risk management 
successfully. Satisfactory 
(Recommendation 1.93 of the 2003 April Report of the Auditor General of Canada, Chapter 1, 
Integrated Risk Management) 


Senior management in departments and agencies should ensure that a departmental risk profile is developed 
‘that identifies and assesses the key risks and challenges of the department and the level of risk it is willing to 
accept. The departmental risk profile should communicate the risks and the tolerance levels to managers and Satisfactory 
staff so they understand the boundaries or limits within which they are expected to manage risk. 


(Recommendation 1.65 of the 2003 April Report of the Auditor General of Canada, Chapter 1) 


Senior management in each department and agency of the government should visibly and actively 
demonstrate clear, sustained commitment and leadership to ensure the successful implementation of 
integrated risk management. This should include such things as assigning adequate resources, monitoring 
the progress of implementation against an approved plan, and demanding risk-based information to 
support strategic decision making. 


" (Recommendation 1.48 of the 2003 April Report of the Auditor General of Canada, Chapter 1) 


Satisfactory 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, and the time that has elapsed since 

the recommendation was made. 

Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, and the time that has elapsed since 
the recommendation was made. 
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1.34 We looked at the measures that the OCG and selected 
departments had taken to identify and address the government’s 

need for knowledgeable and skilled financial professionals. In the 
seven departments, we examined whether the chief financial officers 
(CFOs) have the necessary qualifications, as recommended by the 
OCG’s Guideline on Chief Financial Officer Qualifications. We also 
assessed whether the CFOs’ roles, responsibilities, and accountabilities 
complied with the Policy on Financial Management Governance. 

This included determining whether these individuals had any program 
delivery responsibilities that could conflict with their roles and 
responsibilities as CFOs. 


1.35 We found that the OCG is providing effective leadership in 
enhancing human resource capacity for the financial community 
government-wide. We also found good leadership within departments, 
as the departments have training and development plans in place to 
meet many of their current capacity requirements. However, the OCG 
and departments need to collaborate and develop a strategic approach 
to ensure that succession planning is addressed for the anticipated 
turnover of senior financial executives. 


The chief financial officers and their deputies are professionally qualified 


1.36 The deputy head is responsible under the Policy on Financial 
Management Governance to consider the Guideline on Chief 
Financial Officer Qualifications when appointing chief financial 
officers. We found that all the CFOs in the selected departments met 
the requirements for their position and that the roles, responsibilities, 
authorities, and accountabilities assigned to them complied with the 
Policy on Financial Management Governance. 


1.37 As well, the CFOs in the selected departments provide advice 
to deputy ministers on strategic financial choices. During the audit, 
we noted many examples where the CFOs sit on senior management 
committees to provide strategic advice. For example, the Chief 
Financial Officer at Transport Canada sits on the Department’s senior 
policy, programs, regulatory, and integrated management committees 
to provide input on strategic and operational issues, as well as on the 
financial implications of issues and decisions. Details on the CFO's 
participation in these committees are set out in Transport Canada’s 
new Budget Management Framework that supports the Department’s 
strategic plans and program activity architecture. The framework 
represents a shift from the former practice of allocating and — 
reallocating budgets based simply on organizational structures. 

This demonstrates changes influenced by the CFO's advice. 
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1.38 We also examined whether the CFOs and their deputies in 

the selected departments, as well as in the 15 other large government 
departments (not subject to this audit), met the professional 
accounting designation requirements of the Policy on Financial 
Management Governance. 


1.39 In 2002, when we first reported on this issue, only 33 percent 

of senior financial officers had recognized professional accounting 
designations. In 2010, we found that 82 percent of chief financial officers 
and 82 percent of the deputy chief financial officers in the 22 largest 
departments had these designations. While the Guideline on Chief 
Financial Officer Qualifications requires that at least one of these 
officers hold one such designation, in many departments, both officers 
held designations. This is a significant improvement across the 
government since we first reported on this issue eight years ago. This 
high level of professional qualification provides the large departments 
with the skills and competencies to continue the work to fully meet the 
requirements of the financial management and control policies. 


140 The Office of the Comptroller General has put in place a number 
of measures to address issues related to human resource capacity, 
including 


* promoting professional accounting designations for all mid-level 
financial officers and financial managers across government; 


* continuing the Financial Officer Development Program and 
introducing the Chartered Accountant Student Training Program 
for entry level positions in government; 


¢ developing mandatory training courses for all financial officers; 


* conducting special staffing initiatives, to establish pre-qualified 
pools from which government departments may hire financial 
officers; and 


* organizing forums for financial managers to discuss issues and 
share best practices. 


These initiatives are all positive developments that are expected to 
serve the financial community well into the future. We encourage 
the government to continue these initiatives and build upon them 
as warranted. 
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Human resource capacity in financial management has been enhanced 
across departments 


1.41 We found that all of the selected departments were regularly 
monitoring their human resource capacity in the area of financial 
management and had human resource plans that were in line with 

their corporate plans for financial management. The departments had 
identified, on a timely and thorough basis, their shortages of financial 
officers, gaps in knowledge and competency, and the need to review and 
maintain financial human resource capacity at all levels. All seven of the 
departments had plans for training staff and providing developmental 
experience within the department and, in September 2010, staff were 
to start following the mandatory course curriculum within their 
department. 


1.42. Although most of the selected departments found they could 
attract financial officers, keeping them within a particular department 
was often a challenge. For example, it is not uncommon for an 
accounting professional to change departments to obtain a promotion 
more rapidly. A 2009 Treasury Board of Canada Secretariat survey 
identified the rapid promotion of financial officers as a key cause of 
the knowledge and competency gaps. To address this issue, the 
departments are trying to retain their financial officers by offering 
varied experience and development opportunities within a department 
and establishing career paths. As well, the Office of the Comptroller 
General has drafted a new competency model to help departments 
assess promotion readiness. 


Departments need to work with the Office of the Comptroller General to ensure that 
succession strategies are in place 


1.43. Given that a number of senior financial executives within 
government departments will be eligible to retire in the near future, 

it is important that departments have succession strategies in place to 
prepare for those retirements. We found that departments are at 
various stages in the process of putting succession strategies in place. 
In our view, departments should collaborate with the Office of the 
Comptroller General to ensure they have succession strategies to meet 
future needs. 


144 Recommendation. Selected departments, in collaboration with 
the Office of the Comptroller General, should ensure that succession 
strategies are in place to address upcoming vacancies in key senior 
financial positions. 
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The departments’ response. Agreed. The Office of the Comptroller 
General (OCG) has recently initiated a competitive process for the 
establishment of a pool of candidates at the EX-02 and EX-03 levels. 
The departments will participate in this process and will continue to 
collaborate with the OCG to support efforts to develop a succession 
planning strategy to address upcoming and ongoing vacancies in key 
senior financial positions. 


, 


Accrual-based budgeting © 1.45 = Accrual-based financial information reflects a more complete 
and appropriations == picture of government resources, obligations, financing, costs, and the 

impact of activities than cash-based accounting. Applied to budgeting 
and appropriation of funds, it is intended to provide management with 
complete cost information and allow for more informed decision 
making, such as when evaluating the cost-effectiveness of in-house 
delivery versus contracting for services. Having this financial 
information provides improved transparency and accountability and 
better information for planning and controlling operating and capital 
spending. This, in turn, can give legislators more information to 
consider in holding government accountable for 


¢ the stewardship of public assets, 
* the full costs of programs, and 


* its short-term and long-term financial obligations. 


1.46 Our previous chapters, dating as far back as 1998, have 
highlighted the lack of progress made in implementing full accrual- 
based budgeting and appropriations. The House of Commons Standing 
Committee on Public Accounts has recommended the adoption of 
accrual-based budgeting and appropriations and has repeatedly urged 
the Treasury Board of Canada Secretariat to implement these measures. 


1.47 In our 2006 May Status Report, Chapter 1, Managing 
Government: Financial Information, we recommended that the 
Secretariat promptly complete its study and develop a plan for a 
common approach to accrual-based budgeting and appropriations for 
use by departments. In response, the Secretariat committed to having 
an independent contractor complete a study and to develop an 
implementation plan. Our current audit examined the steps that the 
Secretariat had taken since 2006. 
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The Secretariat needs to decide whether the government will fully adopt 
accrual-based budgeting and appropriations 


1.48 Accrual financial information is used to prepare departments’ 
annual financial results for reporting to Parliament and Canadians. 
However, the Secretariat has not made a clear commitment to fully 
implement the additional approaches of accrual-based budgeting and 
appropriations across the government. In fall 2006, the Secretariat 
presented to parliamentarians the results of a study, by an independent 
contractor, of accrual-based budgeting and appropriation practices of 
other governments. 


149 In 2008, the Secretariat outlined its implementation plan for 
introducing accrual-based budgeting in departments. Departments are 
required to supplement their annual plans, based on cash accounting, 
with future-oriented financial statements that are summarized in a 
separate section in each of their reports on plans and priorities and 
that are attached as a link to the departments’ websites. These future- 
oriented statements are based on a one-year forecast. They are derived 
from departments’ approved operational expenditures and adjusted for 
accrual information and managements’ expectations of the financial 
impact of federal budget announcements and other anticipated future 
events. Beginning with a pilot of 10 departments in the 2009-10 fiscal 
year, these statements will become mandatory for all departments in 
the 2011-12 fiscal year. 


1.50 The Secretariat is of the view that the preparation of future- 
oriented financial statements on an accrual basis and the comparison 
of those amounts against actual results in the historical financial 
statements meets the definition of accrual budgeting. However, the 
Secretariat has not provided guidance or set requirements for using 
the accrual information included in the future-oriented financial 
statements in departmental planning and decision making. Most of the 
selected departments still do not use accrual budgeting information to 
manage on a day-to-day basis, as managers are held accountable for 
managing the funding they receive through appropriations, which is on 
a near-cash basis. 


1.51 The Secretariat plans to evaluate the results of the accrual-based 
budgeting project and continue studying the benefits of extending the 
use of accrual accounting to appropriations in the 2012-13 fiscal year. 
The Secretariat has indicated to us that it has not made a decision on 
whether to implement accrual-based appropriations pending completion 
of the evaluation. It is not clear to us how this future-oriented financial 
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statement project will ensure the implementation of accrual-based 
budgeting throughout government, or how it will address whether and 
how to implement accrual-based appropriations. As well, the Secretariat 
indicated that it wants to further examine international implementation 
experiences and issues on these subjects and professional literature and 
studies of these concepts. 


1.52 We found that, after 10 years of discussing this issue, the Secretariat 
has not decided whether to fully implement the use of accrual financial 
information for budgeting and appropriations in departments 

(Exhibit 1.8). In the meantime, in practice budgets and appropriations, 
with a few exceptions, are based on the cash method of accounting. In 
our view, this does not present the full range of financial implications. We 
continue to believe that accrual-based budgeting and accrual-based 
appropriations would be of benefit to the government. In our view, it will 
promote more informed decision making and transparency by providing 
the full cost of programs and activities of the government and improve 
accountability for operating and capital spending. We encourage the 
Secretariat to undertake and complete its study as planned and decide if 
it will implement accrual-based appropriations. 


Exhibit 1.8 Progress in addressing our recommendation on completing the study of budgeting and 
appropriations and presenting an implementation plan 


sa ee Se te ee = (Aas Rees ON Se POs bee ee 
The Treasury Board Secretariat should promptly complete the 

study of accrual-based budgeting and appropriations at the 
departmental level. It should then present the proposed approach 

for a common basis of planning, budgeting, and reporting, along — 
with an implementation plan, to the House of Commons Standing Unsatisfactory 
Committee on Public Accounts. 


(Recommendation 1.18 of the 2006 May Status Report of the 
Auditor General of Canada, Chapter 1, Managing Government: 
Financial Information) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 
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Conclusion 


1.53 Overall, the Treasury Board of Canada Secretariat, including the 
Office of the Comptroller General (OCG), and the selected departments 
have made satisfactory progress against commitments in many of the 
areas we audited. However, much work remains to be done to implement 
the Policy on Internal Control, and this work is expected to take years to 
complete. As well, the issue of implementing accrual-based budgeting 
and accrual-based appropriations has not been satisfactorily addressed. 


1.54 The OCG has made progress against its previous commitments 
to improve the policy environment, frameworks, and guidance 
pertaining to financial controls across the government. It has done 
this by establishing a policy foundation and providing departments 
with the guidance and tools necessary to successfully implement 
policy requirements. 


1.55 The selected departments have made good progress in 
developing their corporate risk profiles. They are working on 
continually improving the integration of risk management into their 
planning, decision making, and reporting, including enhancing 
communication of risk tolerance within their organizations. 


1.56 The OCG and the selected departments have strengthened their 
human resource capacity in the area of financial management through 
various initiatives, including recruiting and training financial staff. 
Succession strategies for addressing upcoming vacancies in senior 
financial positions need to be better addressed. 


1.57 The Secretariat has not made satisfactory progress in 
implementing accrual-based budgeting and appropriations and has not 
made a clear commitment to do so. 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these 
standards as the minimum requirement for our audits, we also draw upon the standards and practices of 
other disciplines. 


, 


Objectives 


The objective of the audit was to determine whether the Treasury Board of Canada Secretariat, the Office 
of the Comptroller General of Canada (OCG) within the Secretariat, and each of the seven selected 
departments have made satisfactory progress in implementing selected recommendations and addressing 
relevant observations reported in our 2006 May Status Report, Chapter 1, Managing Government: 
Financial Information and our 2003 April Report, Chapter 1, Integrated Risk Management. 


Scope and approach 


The audit covered the Secretariat, which includes the Office of the Comptroller General, as well as the 
following seven large departments, which represent about 60 percent of government expenditures: 


¢ Agriculture and Agri-Food Canada 

+ Department of Finance Canada 

- Foreign Affairs and International Trade Canada 

« Human Resources and Skills Development Canada 
¢ Indian and Northern Affairs Canada 

* Transport Canada 


« Veterans Affairs Canada 


To determine whether senior financial managers had accounting designations, we looked at 15 of the 
largest departments, in addition to the 7 selected for audit. 


We conducted interviews with senicr management and staff in the Secretariat, the OCG, and each of the 
seven departments. We also reviewed and analyzed the supporting documentation that the departments 
and central agencies provided us with. In addition, we confirmed whether chief financial officers and their 
deputies who indicated that they had professional accounting designations were members in good standing 
with their professional accounting bodies. We did not assess specific internal controls in the selected 
departments, nor did we audit the appropriateness and the completeness of the risks identified by 
departments in their corporate risk profiles. 
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Criteria 


The Treasury Board of Canada Secretariat and the Office of the 


Comptroller General (OCG), in cooperation with departments and | 


agencies, develop and implement strategies for improving 
financial controls government-wide, including the following: 


The Secretariat develops policies, directives, and standards for 
establishing and maintaining financial controls. 


The OCG provides guidance, tools, and support to 
departments for the interpretation and implementation of 
Treasury Board policies and related directives and standards. 


¢ The OCG monitors departmental compliance with financial 
management and control policies, directives, and standards, 
and the overall state of financial control government-wide. 


The Programs and Activities of the Secretariat, Management 
Policy Development and Oversight, Treasury Board of Canada 
Secretariat (website: tbs-sct.gc.ca) 


Policy on Financial Management Governance, sections 5.3 
and 6.2—OCG responsibilities, Treasury Board, 2009 


Framework for the Management of Compliance, section 8.2, 
Treasury Board, 2008 


* 2006 May Status Report of Auditor General of Canada, 


Chapter 1, Managing Government: Financial Information, 
Recommendation 1.37 and related response of the Secretariat 


The selected departments assess the effectiveness of their 
system of internal controls for financial reporting and ensure that 
it is maintained, monitored, and reviewed, with timely corrective 
measures taken when issues are identified. 


Policy on Internal Control, sections 5.2.1, 5.2.2, and 6.1.2, 
Treasury Board, 2009 


Policy on Financial Management Governance, section 5.4, 
Treasury Board, 2009 


The Treasury Board of Canada Secretariat provides guidance, 
tools, and support to departments for implementing integrated 
risk management and monitors departmental and agency 
performance in this area. 


The Programs and Activities of the Secretariat, Management 
Policy Development and Oversight, Treasury Board of Canada 
Secretariat (website: tbs-sct.gc.ca) 


Integrated Risk Management Framework, Shared 
Leadership—Suggested Roles and Responsibilities, Treasury 
Board of Canada Secretariat, 2001 


2003 April Report of the Auditor General of Canada, 
Chapter 1, Integrated Risk Management, 
recommendations 1.93 and 1.98 and related government 
responses 


The selected departments have up-to-date corporate risk profiles 
that identify their key risks and risk tolerances, and these are 
communicated to management and staff throughout the entity. 
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Integrated Risk Management Framework, Element 1— 
Developing the Corporate Risk Profile, Treasury Board of 
Canada Secretariat 


Integrated Risk Management Implementation Guide, 
Section 1—Developing the Corporate Risk Profile, Treasury 
Board of Canada Secretariat 


Enterprise Risk Management: Integrated Framework, 
Components of Enterprise Risk Management, Committee of 
Sponsoring Organizations of the Treadway Commission 


2003 April Report of the Auditor General of Canada, 
Chapter 1, Integrated Risk Management, 
Recommendation 1.65 and related government response 
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The selected departments have action plans in place to integrate * Integrated Risk Management Framework, Element 3— 
risk information into strategic planning, decision making, and Practicing Integrated Risk Management, Treasury Board of 
day-to-day operations within reasonable time frames, and Canada Secretariat 

departments regularly assess their progress against these plans. 


Enterprise Risk Management: Integrated Framework, 
Committee of Sponsoring Organizations of the Treadway 
Commission 


ISO 31000: Risk Management— Principles and guidelines 
| 2009, section 4.3.4, ISO. 


. 
e 


2003 April Report of the Auditor General of Canada, 
Chapter 1, Integrated Risk Management, 
recommendations 1.48, 1.56, and 1.81 and related 
government responses 


The Office of the Comptroller General provides leadership and | * Policy on Financial Management Governance, section 5.3— 
direction geared to strengthening financial human resource / Comptroller General’s responsibilities: items 5.3.6, 5.3.7 and 
capacity government-wide. 5.3.8, and section 6.2—item 6.2.4, Treasury Board, 2009 


* The Programs and Activities of the Secretariat, Management 
Policy Development and Oversight, Treasury Board of Canada 
/ Secretariat (website: tbs-sct.gc.ca) 


The selected departments have implemented the Chief Financial | * Policy on Financial Management Governance, section 5.4, 
’ Officer model and have systems, processes, and practices in | Treasury Board, 2009 
place that are designed to ensure that they have the financial and 
risk management human resource capacity to meet their current 


and future requirements. 


¢ Management Accountability Framework, Graphic 3— 
Measures—People, Treasury Board of Canada Secretariat 


The Treasury Board of Canada Secretariat has completed its | ¢ Sixth Report of the Standing Committee on Public Accounts: 
study of accrual-based budgeting and appropriations and is Public Accounts of Canada 2008, 2nd session of the 40th 
developing and implementing an approach for a common basis of | Parliament—Government response (Presented to the House 
planning, budgeting, and reporting. on 19 August 2009) 


Tenth Report of the Standing Committee on Public Accounts: 
Public Accounts of Canada 2009, 3rd session of the 40th 
Parliament—Government response (Presented to the House 
on 18 August 2010) 


2006 May Status Report of the Auditor General of Canada, 
Chapter 1, Managing Government: Financial Information, 
Recommendation 1.18 and the related Secretariat’s response 


| © Treasury Board of Canada Secretariat, Report on Plans and 
Priorities 2009-10—Analysis of Program Activity 


Management reviewed and accepted the suitability of the criteria used in the audit. 
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Period covered by the audit 


The audit covered the period from 1 April 2009 to 30 September 2010. Audit work for this chapter was 
substantially completed on 30 September 2010. 
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Appendix List of recommendations 


The following is a list of recommendations found in Chapter 1. The number in front of the 
recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses 
indicate the paragraphs where the topic is discussed. 


Financial controls 


1.25 The seven audited departments The departments’ response. Agreed. The selected departments 
(Agriculture and Agri-Food Canada, are in the process of carrying out risk-based action plans for 
Department of Finance Canada, completing assessments of financial reporting controls within the 
Foreign Affairs and International Trade current timelines. The departments generally agree to review 
Canada, Human Resources and Skills these timelines in consultation with the Office of the 
Development Canada, Indian and Comptroller General, as well as any timelines they have or will 
Northern Affairs Canada, Transport set for addressing identified gaps and weaknesses in controls. 
Canada, and Veterans Affairs Canada), | Furthermore, the departments will continue to implement 

in consultation with the Office of the programs for ongoing monitoring of their financial controls. 


Comptroller General, should review the 
timelines set for completing their full 
risk-based assessment of the 
effectiveness of their financial reporting 
controls and for addressing identified 
gaps and weaknesses, to ensure that 
they are timely. The departments 
should also implement programs for 
continuous monitoring of these controls 
to confirm their ongoing effectiveness. 


(1.15-1.24) 
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1.26 The Treasury Board of Canada 
Secretariat should continue to review 
the departments’ implementation of the 
Treasury Board Policy on Internal 
Controls to support the effectiveness of 
internal controls for financial reporting 
across government. This should include 
monitoring the departments’ 
established timelines for completing 
their risk-based assessments of these 
controls. The Secretariat should also 
monitor the departments’ actions to 
address significant issues identified as a 
result of their assessments, including 
monitoring that the actions are 
appropriate and timely. (1.15-1.24) 


The Secretariat’s response. Agreed. The Secretariat continues 
to acknowledge the importance of monitoring and strengthening 
internal financial controls across government. The foundational 
Policy on Internal Control, in place since 1 April 2009, is a risk- 
based policy that advocates effective internal control over 
financial reporting in departments. A three-year, phased-in 
approach has been taken to the implementation of the policy 
and includes a requirement for departments to publish the 
results and action plans arising from assessments of the 
effectiveness of internal controls over financial reporting. 


In implementing the Treasury Board’s Policy on Internal 
Control, the Office of the Comptroller General (OCG) supports 
departments and the government's financial management 
community, by reaffirming the policy expectations and 
requirements and by providing tools and guidance. The OCG 
also monitors departmental progress in implementing the policy 
in a variety of ways, including, but not limited to, reviews of the 
annual departmental disclosure on internal controls over 
financial reporting, required under the Policy on Internal 
Control; assessments under the Management Accountability 
Framework; results of audits and evaluations; and special audits 
or horizontal audits that may be commissioned by the 
Comptroller General of Canada. 


Human resource capacity in financial management 


1.44 Selected departments, in 
collaboration with the Office of the 
Comptroller General, should ensure 
that succession strategies are in place to 
address upcoming vacancies in key 
senior financial positions. (1.43) 


Chapter 1 


The departments’ response. Agreed. The Office of the 
Comptroller General (OCG) has recently initiated a competitive 
process for the establishment of a pool of candidates at the EX- 
02 and EX-03 levels. The departments will participate in this 
process and will continue to collaborate with the OCG to 
support efforts to develop a succession planning strategy to 
address upcoming and ongoing vacancies in key senior financial 
positions. 
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What we examined 


Large Information Technology 
Projects 


Main Points 


Large information technology (IT) projects involve more than 
introducing new hardware and software and systems. These projects 
can introduce new processes and practices—new ways of doing 
business—that also need to be successfully implemented before 
organizations can take advantage of potential efficiencies and savings. 
When successful, these projects can change the way that departments 
carry out their work and improve services to Canadians. Because large 
IT projects are complex and costly, they usually involve long planning 
and development times and require significant investments (on 
average, over three years and more than $70 million). 


In 2006, we looked at seven large IT projects and found that only 
two of them met all of our audit criteria for well-managed projects. 
We found that five of the seven projects had proceeded even though 
their business cases were incomplete or out of date or contained 
information that could not be supported. In addition, four of the 
projects were undertaken by departments that lacked the required 
skills and experience to manage them. Although by 2006 the Treasury 
Board of Canada Secretariat had established a framework of best 
practices for managing IT projects, many of the problems identified 
some nine years earlier had persisted. In 2006, we also found that 
limited progress had been made since our previous audit in 1997. 


For this status report, we examined the progress made since 2006 
by the four departments that had not met all of our criteria in 2006. 
We also selected a new project, approved by the Treasury Board 
since 2006, in order to assess the government’s progress in the way 
it approves and manages large IT projects. 


In our 2006 audit, we were denied access to information by the 
Secretariat, which prevented us from completing our review of its 
challenge role in support of oversight of large IT projects by Treasury 
Board ministers. In this audit, we were able to review information that 
demonstrated the role played by the Secretariat. 


Audit work for this chapter was substantially completed 
on 29 October 2010. 
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Why it’s important = The federal government relies on information technology systems to 
provide many programs and services to Canadians. Large IT projects 
are inherently complex, expensive, and risky. Since 2002, the federal 
government has approved funding of $7.5 billion for new business 
projects making significant use of information technology. In the 
four audits we performed since 1995, the projects audited have a 
history of cost overruns and delays, and of not delivering what had 
been planned originally. 


What we found =» Overall, the government has made unsatisfactory progress on 
its commitments in response to our 2006 recommendations. Although 
some improvements have been made, progress has been unsatisfactory 
in the important areas of governance and project business cases. 
Only two of the five projects we looked at, the Temporary Resident 
Biometrics Project and Integrated Revenue Collections systems, met 
most of our criteria for well-managed projects. 


In order to increase the likelihood of success, departments have 
significantly reduced the scope of the projects and considerably 
extended their timelines. In the area of project governance, the 
Expenditure Management Information System and Global Case 
Management System have had important deliverables deferred 
without full analysis of the impacts and costs of not completing these 
projects. In three of the five projects examined, the project business 
cases did not identify measurable benefits or the benefits have not 
been measured. For example, the 2007 business case for the Secure 
Channel no longer quantifies the financial benefits. 


In three of the four projects we examined, departments adequately 
assessed their capacity to manage the projects and demonstrated that 
they were ready to accept the business transformation that came 
with them; in three of the four projects, departments have done an 
adequate job of managing projects risks. 


* The Secretariat has met the commitments it made in response to our 
recommendations we addressed to it. Since 2007, it has completed a 
policy suite renewal, which has led to new policies, standards, and 
guidance that will directly impact information technology projects by 
the end of the five-year implementation period. The Secretariat has 
been actively challenging departments in their preparation of their 
IT project submissions; recently (September 2010) it also submitted 
its first semi-annual report to Treasury Board ministers summarizing 
the health of large IT projects across government. That report 
provided a snapshot of 12 ongoing projects with a total value of 
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$2.4 billion. However, it is too early to assess whether these measures 
will have a significant impact on the management of large IT projects 
by departments. 


The organizations have responded. The organizations agree with all 
of the recommendations. Their detailed responses follow the 
recommendations throughout the report. 
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LARGE INFORMATION TECHNOLOGY PROJECTS 


Introduction 


2.1 Information Technology (IT) projects are critical to maintaining 
and improving the quality and efficiency of the federal public service. 
These projects extend beyond the technical aspects of replacing 
hardware and software. For example, new high-speed electronic 
services may replace high-volume manual transactions, resulting in the 
modernization of current work practices and transforming the way the 
departments or agencies do business. These projects could also involve 
replacing aging applications and infrastructure that are becoming 
increasingly expensive to operate and may pose certain risks to the 
delivery of services to Canadians. 


What we found in previous audits 


2.2 The Office of the Auditor General has examined the 
management of large IT projects (referred to in our previous reports 
as “systems under development”) four times since 1995. In 1995, 1996, 
and 1997, we reported that IT systems under development were 
characterized by 

* inadequate analysis of underlying business issues, 

* inconsistent support from senior management, 

* a lack of experienced resources on project teams, 

* unrealistic time frames, 

* inconsistent involvement and acceptance on the part of users, and 


* a lack of effective monitoring. 


2.3 In Chapter 3 of our 2006 November Report, we revisited large IT 
projects, given the significant investments that the federal government 
was making in this area. We found there had been limited progress 
since our 1997 audit and that, although the Treasury Board of Canada 
Secretariat had established a framework of best practices for managing 
IT projects, many of the problems that we cited in past reports were 
still evident. We also found that, only two of the seven projects we 
reviewed met all of our audit criteria for well-managed projects. 


Events since our last audit 


24 At the time of our 2006 audit, a large IT project was defined as © 
a business project that incorporated a significant IT component, with 
an estimated value that exceeded the project approval limit of a 
department or agency, and therefore required Treasury Board approval 
to proceed. Since 2006, new criteria for setting departmental approval 
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limits for projects, including IT projects, were introduced. These 
criteria assist in ranking a department's overall capacity to manage 
projects and the given IT project’s risk and complexity. 


2.5 In late 2006, the Treasury Board approved the new Policy 
Framework for the Management of Assets and Acquired Services, 
supported by policies approved in 2007. These policies and their 
supporting standards are designed to ensure that the government's 
investments in large capital projects, including [T-enabled projects, 
are managed more effectively and economically. Departments have 
until April 2012 to fully implement these policies and standards. 


2.6 From July 2006 to 2010, the Treasury Board approved 

45 business projects with a significant information technology 
component for a total value of over $3.2 billion. This compares with 
the 75 projects approved for a total value of $3.4 billion from 2002 
to June 2006. During the same period, the Canada Revenue Agency 
approved 35 projects with a significant information technology 
component for a total value of $456 million, compared with 

23 projects for a total value of $431 million from 2002 to 2006. 


Focus of the audit 


2.7. We examined the Secretariat’s role in challenging large IT 
projects and supporting Treasury Board ministers’ oversight, as well as 
its provision of guidance to departments on effective management and 
oversight of their IT projects. 


2.8 We also assessed progress since 2006 of the four projects that had 
not met all of our audit criteria and were not cancelled: 


* Expenditure Management Information System, at the Treasury 
Board of Canada Secretariat 


Global Case Management System, at Citizenship and 
Immigration Canada 


¢ Integrated Revenue Collections, at the Canada Revenue Agency 


¢ Secure Channel, at Public Works and Government 
Services Canada 


2.9 We reviewed one additional project, the Temporary Resident 
Biometrics Project, at Citizenship and Immigration Canada. Since the 
Secure Channel project was completed by 2006, during our current 
audit, we looked at only the progress made in measuring and reporting 
on defined benefits presented in its business case. 
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Treasury Board of Canada 
Secretariat’s roles and 
responsibilities 


During our 2006 audit, the Office of the Auditor 
General did not have full access to information 
related to the Secretariat's challenge role in 
support of Treasury Board ministers’ oversight 
responsibilities, so we could not complete that 
part of our audit. During this audit, we were able 
to review information that demonstrated the role 
played by the Secretariat and have included our 
findings here along with our examination of 
progress against past recommendations. 


Executive project dashboard—A business 
management tool used to visually represent the 
health and status of a project or portfolio of 
projects. Dashboards report on the core project 
metrics affecting the project's overall health and 
benefits realization, including cost, schedule, 
scope, risks, and issues. Their purpose is to 
draw attention to project areas that might 
require corrective action. Executive project 
dashboards are essential for ensuring 
consistency in reporting on the management of 
lT-enabled business projects to senior 
management, primarily at the deputy minister 
and assistant deputy minister levels. 
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2.10 We also reviewed actions taken in relation to the government’s 
commitments in response to the recommendations the Parliamentary 
Standing Committee on Public Accounts made in its 7th and 

4th reports to the House of Commons on 25 February 2008 that were 
relevant to issues within our audit scope. 


2.11 More details about the audit objectives, scope, approach, and 
criteria can be found in About the Audit at the end of this chapter. 


Observations and Recommendations 


212 ‘This section reports on new audit work that deals first with the 
Treasury Board of Canada Secretariat’s challenge role in support of 
Treasury Board ministers’ oversight of large IT projects. Second, it 
reports on the Secretariat’s guidance to departments on effective 
management and oversight of large IT projects. Finally, we reviewed 
the actions taken in relation to the government’s commitments in 
response to recommendations made in 2006 that related directly to the 
Secretariat’s leadership role, and in response to recommendations 
made by the Public Accounts Committee in its 7th report to the House 
of Commons in 2008. 


2.13. The Secretariat makes recommendations and provides advice to 
the Treasury Board on policies, directives, regulations, and program 
expenditure proposals with respect to the management of the 
government's resources. Its other roles include providing leadership to 
help departments and agencies to both adopt best management 
practices and improve their performance. 


2.14 We examined the Secretariat’s governance mechanisms to 
determine whether it has an effective framework for the government- 
wide portfolio of large IT projects under development. 


Central monitoring of large IT projects has been recently enhanced 


2.15 We examined whether the Secretariat, and more specifically its 
Chief Information Officer Branch (CIOB), has instituted measures for 
monitoring and reporting on large IT projects. 


2.16 Since December 2009, the CIOB has periodically received 
executive project dashboards for 12 large IT-enabled business projects 
with a total value of $2.4 billion. These 12 IT projects are a subset of 
the total IT project portfolio; they were selected based on the following 
criteria: significant complexity and risk, significant change in 
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performance, and an estimated budget that exceeded $100 million. 
The dashboards provide a periodic (usually monthly) snapshot of the 
overall health of these projects: 


217 The Secretariat committed to report, on a semi-annual basis, to 
Treasury Board on the overall health of large IT projects. The IT- 
enabled Project Portfolio Status Report is the primary governance 
mechanism for monitoring and advising the Treasury Board on the 
health of large IT projects across government. The CIOB presented 
this report to the Treasury Board for the first time in September 2010. 
It indicated that eight projects totalling $1.9 billion were on track, 
one project needed some course correction, and three projects 
totalling $463 million needed significant course correction. During this 
audit, we examined two of the twelve projects in the report. 


2.18 The Assistant Deputy Minister Executive Project Oversight 
Committee, which was established in July 2009, is chaired by the Chief 
Information Officer (CIO). The committee provides guidance on the 
development of the semi-annual report on IT-enabled project 
oversight and advises the CIO, who in turn uses the report to provide 
advice and recommendations on selected key IT initiatives to the 


Treasury Board. 


2.19 These governance mechanisms that the Secretariat has recently 
implemented are a significant improvement in monitoring the health 
of large IT projects in which the government invests billions of dollars. 
However, it is too early to assess their impact on the success of these 
projects. 


The Secretariat's review mechanisms contribute to the success of large IT projects 


2.20 We examined the systems and practices that the Secretariat has 
designed to increase the likelihood of success for large IT projects at 
the outset, during the Treasury Board submission and approval process. 
This relates to our 2006 recommendation for improved requirements 
for the business cases prepared by departments and agencies. 


2.21 A key practice to enhance the likelihood of success is to review 
large IT project submissions from departments before presenting them 
to the Treasury Board for approval. A submission must include 


* links to new and existing government programs; 
* clear links to Government of Canada IT strategic directions; 
* clear objectives, expected results, and business outcomes; and 


* details of the options considered. 
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2.22 We found that the Secretariat looks for evidence that 
departments are using effective project management and oversight 
practices that are suitable for large IT projects. We also noted that the 
Secretariat has provided comprehensive guidance to departments in its 
Business Case Guide and Outcome Management Guide. 


2.23 We found that the CIOB reviews IT project submissions to 
determine whether the “conditions for success” are apparent in the 
management framework that the department will use for the proposed 
project. CIOB officials probe and challenge the project proposal to 
determine whether there is 


* aclear business justification; 
* aclear and reasonable scope; 
* a suitable governance framework; and 


* reasonable estimates, decision points, and independent reviews. 


2.24 Inthe submissions we reviewed, we found documentation of 
the Secretariat’s challenges in the form of emails and meeting notes 
but no formal procedures on how it performs its review and challenge 
function. The Secretariat informed us that the challenge process is 
largely done orally and the extent of the review depends on the nature 
of the proposal, and that various policies apply on a case-by-case basis. 


2.25 Based on industry-leading practices, the CIOB has created 
additional tools and guides to assist the departments in improving IT 
project management. Although it is not mandatory for departments to 
use these tools, the CIOB uses them in its review of project submissions. 
The Secretariat's program sectors seek advice on IT issues from the 
CIOB, and both the CIOB and the program sectors communicate 

with departments to offer them advice on the proposed management 
framework of their IT projects. The CIOB has implemented an 
independent review program that uses a gating framework designed to 
enable departments to improve their investment outcome. It strongly 
encourages departments that are undertaking complex, high-risk IT 
projects to take advantage of this program early in the planning process. 
The departments use these independent reviews to gather objective 
evidence to validate their options and costs, assess the feasibility of their 
project plans and schedule, and assess the capacity of the organization 
to take on the project. 


2.26 Secretariat officials told us that its challenge function is limited 
to a review of compliance with government policy because the deputy 
minister of a department is responsible for ensuring the completeness 
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and accuracy of the submission. The deputy minister in turn relies on 
the recommendation of the senior financial officer to proceed with a 
project submission to the Treasury Board. Since December 2007, the 
senior financial officer must attest to the deputy minister that, in his or 
her professional opinion, the information in the submission is fairly 
presented, proper analysis has been carried out, and due diligence has 
been exercised. 


2.271. We observed that, since the clarification of the senior financial 
officer’s responsibility, the Treasury Board submissions and supporting 
business cases for the projects that we reviewed have more conservative 
plans and reduced scope than before. We also noted that the 
departments have undertaken independent reviews of their project . 
plans and organizational capacity to assist them in performing their due 
diligence before submitting their project proposals to Treasury Board. 


2.28 Based on these findings, the Secretariat and its CIOB have made 
satisfactory progress against our recommendation for improved 
requirements for business cases (Exhibit 2.1). 


Exhibit 2.1 Progress in addressing our recommendation on business cases 


The Treasury Board Secretariat should improve the requirements 
for sound business cases prepared by departments and agencies to 
ensure that they include, at a minimum, precise and measurable 
objectives; a full analysis of options, benefits, costs, and risks; and 
an implementation plan. 


(Recommendation 3.90 of the 2006 November Report of 
the Auditor General of Canada, Chapter 3, Pg Information 
Technology Projects) 


Satisfactory 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


The Secretariat has improved its policy framework for the overall direction 
of large IT projects 


2.29 We recommended in 2006 that the Secretariat ensure that 
departments demonstrate their capacity to undertake large IT projects. 


2.30 In 2006, Treasury Board approved the Policy Framework for the 
Management of Assets and Acquired Services, which sets overall 
direction for large projects. The framework is made up of policies and 
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standards that seek to ensure that government projects provide value 
for money and are managed effectively. 


2.31 The Secretariat is responsible for assessing whether departmental 
deputy ministers comply with the framework’s policies and standards and 
for advising the Treasury Board on the need for additional mechanisms if 
needed. Full compliance with these policies and standards is not required 
until 2012, five years after they came into effect. 


2.32. The Policy on Investment Planning, which is part of the new 
framework, requires departments to submit an investment plan covering 
a minimum five-year period. The Policy on the Management of Projects, 
which is also part of the new framework, requires departments to submit 
an Organizational Project Management Capacity Assessment. The 
result of the capacity self-assessment is one of the new measures used to 
arrive at the project approval limit for the department. The Secretariat 
has committed to completing a high-level review of the completeness 
of the department’s self-assessment and does not examine the 
department’s project management capacity for each of its capital 
projects, which Treasury Board policy indicates is the role of deputy 
ministers. At the time of this audit, Public Works and Government 
Services Canada was the only department among those we examined 
that had completed the self-assessment of its organizational project 
management capacity and submitted it to the Secretariat for review. 
However, progress in addressing our recommendation is satisfactory 
given that a tool is in place and some departments have started to 
complete these self-assessments (Exhibit 2.2). 


Exhibit 2.2 Progress in addressing our recommendation on organizational capacity 


Recommendation | Progress 


a 


Before recommending that the Treasury Board approve an 
IT project, the Treasury Board Secretariat should 


* ensure that the departments and agencies submit their 


analyses of organizational capacity, and ; 
Satisfactory 


¢ validate the content and approach of these assessments. 


(Recommendation 3.99 of the 2006 November Report of 
the Auditor General of Canada, Chapter 3, Large Information 
Technology Projects) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 
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The Secretariat implemented an action plan, improved its guidance, 
and improved reporting 


2.33 In its 7th report to the House of Commons in 2008, the Public 
Accounts Committee (PAC) made recommendations based on the 
results of our 2006 chapter on large IT projects. The government 
agreed to act on these recommendations. 


2.34 PAC recommended that the Secretariat develop an action plan 
to address our recommendations. The Secretariat submitted its action 
plan to PAC and has implemented it, including the completion of the 
Policy Framework for the Management of Assets and Acquired 
Services and the new Policy on the Management of Projects. It has also 
made significant improvements to its guidance. 


2.35 PAC also recommended that the Secretariat direct departments 
and agencies to provide financial and performance information in the 
reports they submit to Parliament on IT projects that are expected to 
cost over $10 million, including 


* original and current estimated total costs, 
* costs incurred to date, 
* the expected completion date, and 


¢ the intended outcomes. 


2.36 In response, departments have included two tables in 

their 2009-10 departmental performance reports, entitled Status 
Report on Major Crown/Transformational Projects and Status Report 
on Projects Operating with Specific Treasury Board Approval. 


Progress in key management areas 2.37 This section presents our findings of the progress the government 
made against the commitments it made to address recommendations in 
our 2006 Report to Parliament. These commitments related to four key 
management areas—governance, business cases, organizational capacity, 
and project risk management. We assessed the progress made since 2006 
of four large IT projects: Expenditure Management Information System, 
Global Case Management System, Integrated Revenue Collections, and 
Secure Channel. We also reviewed an additional project, the Temporary 
Resident Biometrics Project, which was implemented since our 
2006 audit. Since the Secure Channel project was completed in 2004, 
we examined only whether defined benefits noted in its business case 
have been achieved and measured. The project history in Exhibit 2.3 
shows that departments have significantly reduced the size and scope of 
certain projects. For all the projects we reviewed, budgets have increased 
by approximately $649 million since their inception, and time frames 
have also increased significantly. 
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Governance structures remain problematic for two of four projects we examined 


2.38 Governance at the project level focuses on delivery of business 
change at an affordable cost with an acceptable level of risk. Treasury 
Board policies governing investment planning and projects stipulate 
that the sponsoring department or agency of an IT investment project 
is responsible for establishing and controlling the costs and risks and 
for ensuring that the project benefits the organization and meets a 
defined business need. The sponsoring department is responsible for 
ensuring the appropriate capacity will be in place to manage and 
deliver the proposed project and to show that it is making a sound 
investment decision—based on objective, unbiased information. 

A sound governance framework at the project level is critical to 
fulfilling these responsibilities. 


2.39 In 2006, we examined whether the processes departments used 
to approve and manage seven selected large IT projects increased their 
likelihood of success. We found that four of the projects had deficient 
governance frameworks. Even though the governance structure was 
defined for all projects, we found that not all responsibilities were 
carried out and not all key business issues were reported and resolved. 


2.40 As previously noted, the Treasury Board of Canada Secretariat has 
put in place new Treasury Board policies and tools to assist departments 
in approaching IT projects from a business perspective rather than 
simply a technology perspective. This business perspective would place 
appropriate attention on planning the project, managing the necessary 
transition, and involving all key stakeholders. We looked at whether 
departments have implemented this approach for their projects. 


2.41 Expenditure Management Information System (EMIS). 

In 2000, the Secretariat initiated EMIS to replace five legacy systems 
that were originally created in the mid-1980s to prepare the annual 
budgetary estimates for Parliament's approval. As we reported in 2006, 
the project’s scope was expanded significantly to include the building 
of a new comprehensive system that would provide an overall view of 
the government’s financial performance. By 2006, the newly defined 
project had four phases and its budgeted expenditures had gone from 
$16.2 million to $53.7 million. In 2007, the Secretariat scrapped the 
two components it had built between 2000 and 2006 and started over, 
at a cost of $34.5 million, because the components were found to be 
unreliable. Since 2007, the Secretariat has completed the first 

two phases of the project and has deferred indefinitely the last 

two phases. 
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2.42 In 2006, we reported that this project had no clear governance 
and accountability structure in place. As well, there was no monitoring 
of key performance indicators or status of deliverables. In our current 
audit, we found that, since 2006, most of the governance weaknesses 
have been addressed. A clear governance structure was in place, 
including regular dashboard reporting to a senior steering committee 
that included financial updates and information on risk and progress. 
However, we found little evidence that senior management had 
analyzed the risks and impact of deferring the last two phases 
indefinitely. Without building the remaining functionalities, the 
Secretariat will be unable to achieve the overall project’s objectives. 


2.43. In its 4th report to the House of Commons in 2008, the Public 
Accounts Committee (PAC) recommended that the Secretariat 
provide an update on the implementation of EMIS and that it appoint 
a blue-ribbon panel to examine financial information systems with a 
goal of improving spending decisions. The government agreed to have 
the Secretariat report to PAC on the status of EMIS in July 2008. 
However, when it did, it did not report the full extent of delays in 
completing the second phase or the possibility of deferring the 
remaining two phases indefinitely. In lieu of appointing a blue-ribbon 
panel, the government indicated that the Secretariat would engage 
external advisers to provide technical and strategic advice on EMIS. 
We found that the technical advice, related to the subsequent phase of 
EMIS, was provided and that, more recently, an advisory committee 


was created to provide strategic advice. 


2.44 Recommendation. The Treasury Board of Canada Secretariat 
should analyze the risks and impact of deferring the last two phases of 
the Expenditure Management Information System (EMIS) indefinitely 
and take the necessary action consistent with the Secretariat’s 
priorities and future objectives for the project. 


The Secretariat’s response. Agreed. The Secretariat will complete the 
documentation of the decision to cancel the last two phases originally 
planned for the EMIS project by April 2011. The risks and benefits of 
the decision have been assessed, and the decision is consistent with the 
Secretariat’s priorities and objectives. 


2.45 Global Case Management System (GCMS). In 2000, 
Citizenship and Immigration Canada (CIC) initiated the GCMS with 
the business objective of improving security and efficiency in the delivery 
of immigration and citizenship activities. The GCMS was to replace 

14 legacy and aging systems that did not work well together and placed 
CIC in a position of serious risk. GCMS is based on one integrated 
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case-management system that provides an end-to-end view of the client 
continuum from first contact to final disposition. The project’s original 
cost was $195 million, and it was to be completed by 2005. 


2.46 In 2006, we found that the project’s governance structure was 
not well established. This was further confirmed by an independent 
review that was conducted in the same year, which noted that the 
governance mechanism relied on the efforts of individuals and that the 
project management office lacked the authority to challenge the state 
of deliverables. It recommended that a suitable governance process be 
put in place for both CIC and the Canada Border Services Agency 
(CBSA) to use GCMS as a shared resource. 


2.41 CIC has made progress in addressing weaknesses in the GCMS 
governance. For example, it has established clear lines of accountability 
by naming the Assistant Deputy Minister, Corporate Services, as project 
leader; it has appointed an experienced business director to represent 
the interests of CIC and CBSA; and a senior interdepartmental 
committee exercises project oversight through, among other things, 
the review of executive project dashboards and project summaries, and 
the monitoring of major issues and risks. 


2.48 Since our 2006 audit, we found that CIC significantly improved 
its ability to pursue the delivery of GCMS, as seen in the current phase 
of the project, Release 2, which focuses on immigration processing. 
This phase has received close scrutiny by senior CIC committees and 
has been reporting regularly on its progress. As well, the deployments 
to date have been successful. Current project dashboards show that 
the current phase will be delivered on time and slightly below budget. 


2.49 The GCMS business objectives that were first identified in 2000 
were reconfirmed in several key documents throughout the life of the 
project. However, the current phase is set to end on 31 March 2011, 
and a significant amount of work still remains. CIC has assessed 

that about 25 to 31 percent of GCMS functionalities will still not 

be developed. The GCMS work that will remain is mainly related 

to extending the GCMS capability for domestic enforcement and 
immigration services at ports of entry and inland offices. As a result, 
many of the legacy systems that were to be replaced will continue 

to be used after project closing. For example, the Field Operations 
Support System (FOSS), a 30-year-old system critical to the 
National Immigration Program, which is considered high risk due 

in part to integration issues, will not be decommissioned until 


the 2015-16 fiscal year. 
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2.50 Iwo independent reviews, in 2007 and 2009, recommended that 
CIC continue to focus on future requirements and that it start planning 
subsequent phases of GCMS before completing the current phase. 

As well, in its 2009-2012 IMTB (Information Management and 
Technologies Branch) Integrated Business and Resource Plan, CIC 
stated that it would produce a business case and funding proposal for 
the remaining work. Although discussions have occurred at senior 
levels, they have not clearly addressed the issue of the outstanding work 
and its impact on the business objectives. As well, some documents 
have been initiated but they are still in draft form, have not been 
approved, and include figures that are not supported with analysis. 


2.51 Recommendation. Citizenship and Immigration Canada should 
prepare a plan to address the costs and risks related to the remaining 
work to be done after completion of Release 2 of the Global Case 
Management System (GCMS) in order to achieve the project’s overall 
business objectives. 


The Department’s response. Agreed. Over time, the implementation of 
the GCMS project has been adapted to meet the changing business and 
operational needs of Citizenship and Immigration Canada (CIC). The 
overall project plan was amended in 2008 to focus on overseas 
immigration processing. GCMS Release 2 will be successfully delivered 
worldwide by March 2011, fulfilling the scope of the Major Crown 
Project. CIC is preparing a five-year Roadmap to achieve its overall 
business needs, which have evolved since the original business objectives 
of GCMS were established. As we move forward, we will continue to 
build on the GCMS platform and address any risks related to the 
changes to the original scope. The first year of this Roadmap will be 
funded through the CIC Business Planning and Investment Planning 
process, and the next deployment is scheduled for summer 2011. 

As business priorities shift and change quickly, the yearly process of 
approval will be followed for GCMS over the coming years. A portion 
of immigration inland processing will be implemented in the Case 
Processing Centre Mississauga and CIC Etobicoke in March, with 
minimal IT changes to the system. 


2.52 Integrated Revenue Collections (IRC). In 2001, the Canada 
Revenue Agency (CRA) initiated IRC to help prioritize and allocate 
collections work according to level of risk. This project had an original 
budget of $2.5 million with an expected completion date of 2004; 
however, it did not progress for many years. By fall 2005, senior 
management decided to put the project on hold while a new business 
case was developed. By that time, the project had spent $13 million 
and had few deliverables. 
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2.53 In 2006, we reported that several measures had been taken to 
improve the project’s governance. For example, the CRA established a 
new senior committee, the Resource Investment Management 
Committee (RIMC), in early 2006, to oversee the management and 
progress of major IT projects, including IRC. This committee is chaired 
by the Commissioner, and senior executives representing internal 
stakeholders from other sectors of the Agency are members. Its role 

is to provide assurance that all resource and investment decisions 
support and are fully integrated with Agency direction and strategies. 


2.54 Since 2006, RIMC has monitored IT projects such as IRC 
through progress reports and other similar presentations. It also 
requires project managers to develop a full business case and project 
sponsors to identify the expected benefits and, after implementation, 
confirm that they have realized these benefits. Other governance 
improvements include establishing a project management office and 
dividing the IRC project into more manageable components. 


2.55 Temporary Resident Biometrics Project. Led by Citizenship 
and Immigration Canada (CIC), the biometrics project aims to 
enhance security by capturing photographs and fingerprints of 
temporary resident applicants to ensure individuals posing safety 

and security threats do not gain access to Canada. The project is 
dependent on multiple stakeholders, including the Canada Border 
Services Agency (CBSA), which will verify travellers’ identities against 
previous captured biometrics, such as fingerprints, at selected ports of 
entry. The Royal Canadian Mounted Police (RCMP) will cross- 
reference fingerprints against criminal records databases as part of its 


Real-Time Identification (RTID) Project. 


2.56 The project is governed by interdepartmental committees 
composed of senior managers from CIC, the RCMP and CBSA, as well 
as representatives from the Secretariat and other departments. These 
committees meet regularly and provide a forum for members to raise 
and address issues. Our review of committees’ minutes and planning 
documentation reveals that the biometrics project is a high-risk project 
that has been in “red status” (meaning that significant course 
correction may be required) since April 2009, when projects began to 
maintain dashboards. 


2.57 The biometrics project has been in red status for various reasons. 
First, the project received only a portion of required funding under 
Budget 2008: $180 million instead of the required $208 million. 
Second, the project’s revised scope, designed to enable delivery for 
$180 million, was still being finalized in December 2010. Third, the 
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project is approximately a year behind in receiving approval and 
issuing a request for proposal. We found that CIC has mechanisms 
in place to alert senior management and the Secretariat to these 
problems in a timely manner, and that delivery partners have used 
this information in their decision to continue with the project. 


2.58 In summary. We found that overall progress on the departments’ 
efforts to place more attention on planning large IT projects, managing 
the necessary transition, and involving all key stakeholders has been 
unsatisfactory; two of the four projects, the Global Case Management 
System and the Expenditure Management Information System, have 
significant governance issues (Exhibit 2.4). 


Exhibit 2.4 Progress in addressing our recommendation on improving project governance 
and accountabilities 


De Tae Ts ee S4 FF Wer or aye 7 


Government departments and agencies should improve their 
internal quality reviews of IT projects. Senior departmental 
executives should review the key decision documents that are Satisfactory 

produced to support the IT project and ensure that the analysis is | (Biometrics, IRC) 


thorough and supportable before signing off on the submission. 
Unsatisfactory 
(Recommendation 3.61 of the 2006 November Report of (GCMS, EMIS) 


the Auditor General of Canada, Chapter 3, Large Information 
Technology Projects) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Most business cases do not clearly identify and measure benefits 


2.59 A business case is the foundation for sound investment 
decisions. For IT projects, the business case explains the rationale for 
the project and the results that the project is expected to produce to 
meet an organization’s business needs. It also provides an analysis of all 
the costs, benefits, and risks associated with the proposed investment. 
Furthermore, the business case must define what constitutes success, 
which will be used to measure the project’s success later on. 


2.60 In 2006, we examined seven projects to determine whether 
departments had clearly defined the business needs they expected to 
fulfill. We reported that, for five of the projects, the business cases were 
incomplete, out of date, or contained information that could not be 
supported. In this audit, we examined whether the business cases have 
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improved for four of those projects, as well as for the new Temporary 
Resident Biometrics Project. 


2.61 Expenditure Management Information System (EMIS). 

In 2006, we reported several deficiencies in the Expenditure 
Management Information System’s original 2004 business case, 
notably, that it contained incomplete or unclear business objectives 
and timelines and did not include an options analysis. 


2.62 In 2006, the Secretariat approved a new business case that 
reduced significantly the scope of this project. The new business case 
contained a detailed options analysis and it set out clear business goals 
and measurable performance objectives and planned results, which 
were measured against actual outcomes. Key stakeholders were 
involved in the decisions made on the reduced scope, and it was 
completed on time and within budget. 


2.63 In July 2008, the government agreed to provide the Public 
Accounts Committee (PAC), by the end of 2008, with a close-out report 
summarizing lessons learned from the original EMIS project; it said in its 
response that these were taken into account in the planning and delivery 
of the newly scoped project. The report would indicate how the 
Secretariat would apply lessons learned, including the deficiencies in 
the business case that were identified in our 2006 audit. Although the 
Secretariat has completed this report, it has not provided it to PAC. 

In addition, the government agreed that it would make project close-out 
reports, as well as their filing with CIOB, mandatory for all large IT 
projects. We noted that the Secretariat advised PAC in March 2010 that 
its project evaluation guidelines indicate that close-out reports are to 
be prepared within three months of completion of a project. However, 
the use of these evaluation guidelines is not mandatory. 


2.64 Global Case Management System (GCMS). In 2006, we 
reported that, since project initiation in 2000, Citizenship and 
Immigration Canada (CIC) struggled with a number of issues, 
including changes in project scope, difficulties accessing funds, a lack 
of people with the required skills, and delays in procurement. We also 
found that the original business case did not present a detailed analysis 
of options and did not quantify the expected business outcomes and 
benefits. During that time period, the budgeted cost escalated from an 
original budget of $195 million to $243 million; however, there was no 
update of the original business case to reflect these significant changes. 
As a result, the implementation of the citizenship module used most of 
the original budget of $195 million and took as much time as the whole 
project had been expected to take. 
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2.65 CIC issued a revised business case in 2007, which addressed 
scope and capacity issues and, more specifically, focused on the 
immigration line of business. It made the case for another budget 
increase of $144 million for a total of $387 million and a significant 
reduction of scope. We found that the business case included an 
improved options analysis; however, it reiterated almost the same 
expected business benefits as those identified in the original business 
case in 2000. Again, it did not quantify them to make them ultimately 
measurable. In January 2008, the Secretariat, in its challenge role, 
suggested to CIC that it needed to better define the project business 
benefits and provide measures. However, CIC has not acted on this 
advice and has not prepared a benefit measurement plan. As a result, 
it will be difficult for it to provide performance information that shows 
whether the GCMS will achieve the intended business benefits. 


2.66 In 2009, CIC updated the business case again to reconfirm the 
relevance of the direction taken in 2007. Although it also mentioned 
the business benefits at a high level, it did not quantify them or define 
measures. In addition, the significant cost avoidance from the 
replacement of the 14 disparate legacy systems by one integrated 
system mentioned in the original business case was no longer 
mentioned after 2007. 


2.67 Integrated Revenue Collections (IRC). In 2006, we reported 
that IRC was one of several projects in which the business case was 
incomplete and out of date. 


2.68 The Canada Revenue Agency prepared a revised business case 
in 2007 and recommended it for approval by the Resource and 
Investment Management Committee (RIMC). We found this business 
case to be complete, with the exception of the section on benefits 
realization, which required more measurement detail. The main 
benefit identified in the business case was an estimated $700 million 
in annual savings from program effectiveness, improved accounts 
receivable, and revenue collection productivity gains. Of this total, 
$200 million in annual benefits was specifically identified as the result 
of the implementation of the first phase of the project, due to be 
completed by June 2012. However, the business case lacked details on 
how these results will be achieved and contained no information on 
the remaining $500 million. 


2.69 In late 2009, a benefits measurement plan for Phase | was 
prepared and presented to RIMC for approval. The RIMC did not 
approve the plan at that time, as the plan did not outline the benefits 
resulting from the implementation of the first phase of the IRC project. 
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A revised version has since been prepared and was to be presented to 
RIMC in December 2010. This revised plan contains more details on 
how the $200 million in annual gains will be achieved, including 
benchmark information and data collection. It also provides a timeline 
of when the project will start to realize the benefits. However, this 
timeline still lacks details on when this target will be achieved and 
how progress will be measured. In addition, as a result of a change in 
investment decisions for the second phase of IRC, the remaining 
$500 million in annual benefits is no longer applicable and any future 
benefits are to be defined at each additional phase. 


2.70 Secure Channel. The Secure Channel is a complex, innovative, 
multi-departmental project that provides Canadians and businesses with 
secure and private access to online government services. This common 
secure infrastructure ensures that Canadians can use a single window 
to conduct online transactions safely and effectively. In 1999, the 
Secretariat initiated the Secure Channel and was responsible for its 
oversight, strategic direction, and decision making. Public Works and 
Government Services Canada (PWGSC) was responsible for project 
management and also became responsible for operations in 2004. 

The project’s development had a total cost of $377 million, of which 
approximately $285 million can be attributed to the development of the 
authentication (epass) service, one of five main services that the Secure 
Channel provides. The project's 2005 business case identified benefits 
and related targets to be achieved over a five-year period, and included 
projections on the average Internet transaction volumes, and significant 
cost savings and cost avoidance. As well, the business case identified 
program administration benefits estimated at $5.2 billion (unaudited), 
which could not be achieved without this secure infrastructure. 


2.71 Since our 2006 audit, the Secretariat has made the use of all the 
Secure Channel Services mandatory, by amending the Treasury Board 
Common Services Policy. It did this to ensure that federal departments 
would have to use Secure Channel services and thus these services 
could be funded on a cost-recovery basis. The cost of operating the 
Secure Channel from October 2004 to March 2010 was $483 million. 
It is estimated that another $134 million will be spent between 

April 2010 and December 2012. This will bring the total expenditure 
to $617 million, of which the epass service component is $367 million. 


2.72 Human Resources and Skills Development Canada (HRSDC) 
and Canada Revenue Agency (CRA) programs accounted for over 
90 percent of the epass service annual volumes. Because of this, 
PWGSC required that they make binding commitments on the 
number of projected transactions and used these to calculate the 
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amount that each department would be required to pay on a yearly 
basis. These costs were binding and payable whether or not HRSDC or 
CRA achieved their projected transaction volume levels. 


2.73 The HRSDC My Service Canada Account (MSCA) started using 
the epass service in November 2006. Soon after, it started to encounter 
performance and system availability issues during periods of high 
transaction volumes. As a result, HRSDC stopped using the epass 
service for its MSCA in February 2007. The MSCA generates, on 
average, 12 million transactions annually. PWGSC indicated that it had 
addressed these issues by August 2007 and formally advised HRSDC in 
February 2008. HRSDC then reintegrated the MSCA in May 2008 


when service disruption to Canadians would be kept to a minimum. 


2.14 In its initial letter of intent in November 2006, CRA indicated its 
concern that the cost of the epass service was substantially more than 
its costs to operate and maintain its own in-house solution that met its 
security and operational requirements. Furthermore, when determining 
its service volume commitments in 2006, CRA decided to exclude 
electronically filed tax returns, which generate, on average, 

12 million transactions annually, as including them was not cost 
effective. The Agency has recently built an in-house service that 
provides it with a different level of security and privacy that meets its 
requirements. CRA stopped using the epass service as of October 2010. 


2.75 Inearly 2007, the Secretariat approved a revised Secure Channel 
business case. This business case focused on the technical business 
requirements, such as security and privacy, timely availability, a single 
window, and interoperability between departments, programs, and 
services. The business case also indicated that there would be 
operational benefits related to departmental take-up and transaction 
volumes, as well as financial benefits consisting of cost savings and 


cost avoidance. 


2.16 The 2007 business case does not identify specific measures, 
timelines, and targets for the technical requirements or operational 
and financial benefits. This is in sharp contrast to the previous business 
case in 2005, which identified the business benefits with specific 
targets and measures. Finally, there is no benefit measurement plan 
and no indication of who would be responsible for measuring benefits. 
Given the experience with the two largest user departments on its 
most significant service offering, it would have been important to 
clearly indicate in the new business case a target take-up rate or the 
level of financial benefits expected to be achieved. 
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2.77 The Secretariat’s Chief Information Officer Branch (CIOB) is 
currently considering a new strategy for the provision of authentication 
services to departments and agencies after the current epass service 
contract ends in December 2012. As part of this review, the CIOB has 
acknowledged that the current epass single architecture, with the 
bundling of security services, resulted in a solution set that was 
perceived to be excessive and too costly for most applications and that 
the take-up by departments was not achieved as originally estimated. 
The CIOB has developed a new business case that outlines options and 
cost estimates for a more flexible authentication solution for 
departments and agencies. 


2.78 Inits 7th report to the House of Commons in 2008, the Public 
Accounts Committee recommended that PWGSC conduct a cost- 
benefit analysis of the continued use of Secure Channel 

by 31 December 2008, and that PWGSC also provide meaningful 
results-based information in its annual departmental performance 
report (DPR). In June 2009, the government agreed to both 
recommendations and provided the Committee with its SC/EPASS 
Evolution Analysis Report, which gives an update on progress with 
regards to cost recovery of all Secure Channel services. PWGSC has 
also provided on an annual basis a number of performance measures 
relating to the usage of the Secure Channel epass service in its DPR. 


2.19 We reviewed these performance measures and found they 
included the number of programs and departments using the service, 
transaction volumes, per transaction cost, and capacity use, as well as 
the number of epasses issued to businesses and Canadian individuals. 
However, these measures lacked context. There were no targets 
against which this information can be compared to assess performance. 
For example, the last report stated that 8.2 million epasses have been 
issued to date. This was represented as showing that Secure Channel 
has achieved full capacity. We note that there is no discussion of how 
this compares with the architectured capacity of 30 million epasses. In 
addition, the actual number of transactions between April 2007 and 
December 2009 of 47.9 million was not compared with the minimum 
transaction volume commitment of 93 million made for that period. 
As well, this information was not compared against the annual 
estimated capacity of 45 million transactions per year, which an 
independent review in 2005 had stated was needed for the epass 
service to be cost effective. It is difficult to properly assess the 
performance of the Secure Channel service without meaningful 
comparisons against plans and targets. 
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2.80 ‘Temporary Resident Biometrics Project. Citizenship and 
Immigration Canada estimated the initial cost for implementing the 
biometrics project, including the CBSA and RCMP components and 
interfaces, at $208 million. However, we note that the 2008 Budget 
did not provide most of the funding for the CBSA component and 
approved funding of only $180 million. Because of this funding 
shortfall, CBSA indicated that it would not be able to complete some 
of its development activities and would have difficulty supporting 
ongoing operations of fingerprint enrolment and systematic 
verification at all ports of entry. The biometrics project’s 2009 business 
case took into account the reduced funding; however, it did not clearly 
articulate what functionalities would be reduced and what benefits 
would not be realized. 


2.81 In spring 2010, CIC made a strategic change to its service 
delivery model, based on lessons learned from other jurisdictions. This 
changed the project costs significantly as it was going from a user-pay 
model to one where the Government of Canada enters into contracts 
with third-party service providers and pays for the service. While this 
approach enhanced oversight and control, it also shifted the financial 
burden from applicants to the Government of Canada. 


2.82 Because of the revised funding method and change in strategic 
direction, CIC developed a new business case in September 2010, 
proposing a reduced-scope option for the project. Under this option, 
CBSA will be conducting only discretionary fingerprint verification at 
selected ports of entry. This reduced scope option was still being 


finalized in December 2010. 


2.83 We found that the September 2010 business case contains 
well-defined objectives and an options analysis, and represents an 
effort on the part of CIC to develop a viable and focused scope for the 
project where deliverables can be achieved. The project team has also 
begun to develop a framework for monitoring performance. While the 
framework does not contain performance targets or timelines, it does 
include key performance indicators, data sources, and mechanisms for 
reporting performance. 


2.84 In summary. The Secretariat has met its commitments and 
developed and published business guidance to be used by departments 
in preparing their business cases (paragraph 2.22 and Exhibit 2.1). 
However, we found that three of the five projects we examined in 

this audit—GCMS, IRC, and Secure Channel—continue to have 
deficiencies regarding identification and measurement of benefits 


(Exhibit 2.5). 
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Exhibit 2.5 Progress in addressing our recommendation on business cases 


Before seeking effective project approval, departments and 


agencies should prepare a business case that includes, at a Satisfactory 
minimum, precise and measurable objectives; a full analysis of (EMIS, 
options, benefits, costs, and risks; and an implementation plan. Biometrics) 
(see Recommendation 3.89 of the 2006 November Report of Unsatisfactory 
the Auditor General of Canada, Chapter 3, Large Information (GCMS, IRC, SC) 


Technology Projects) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


2.85 Recommendation. When departments and agencies develop a 
business case, they should include a benefit measurement plan that 


* contains specific quantifiable benefits including clear benchmarks, 
* assigns responsibilities for delivering identified benefits, and 
* includes a process for measuring and reporting benefits. 


The Canada Revenue Agency’s (CRA’s) response. Agreed. 
The Agency believes that since the end of the audit (31 October 2010) 
it has addressed the recommendation in the following manner: 


As noted by the Auditor General, a benefits measurement plan for 
Integrated Revenue Collections (IRC) Phase | had been completed by 
the CRA in 2009. The benefits measurement plan for IRC Phase | was 
further refined in 2010 to reflect annual gains to be achieved, 
benchmark information, and a data collection strategy, including when 
benefits would start to be realized, and who would be responsible for 
the delivery of the identified benefits. The 2010 revised plan was 
recently approved and work is ongoing in respect of measuring and 
reporting on the benefits. In fact, the approval process in place at the 
Agency requires that a benefits realization plan be prepared as part of 
business cases for large IT-enabled projects, which is an established 
best practice. 


Citizenship and Immigration Canada’s (CIC’s) response. Agreed. 
Future projects will leverage the CIC Performance Measurement 
Framework and Operational Baselining Strategy in order to quantify, 
measure, and report on benefits. 
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Public Works and Government Services Canada’s (PWGSC’s) 
response. Agreed. PWGSC is committed to adhering to applicable 
Treasury Board policies, as well as Treasury Board of Canada 
Secretariat Chief Information Officer Branch guidelines and standards 
on the development of business cases. In cases where PWGSC is 
responsible for the implementation of large, horizontal IT projects with 
government-wide or multi-departmental impacts, the Department will 
continue to work cooperatively with the Secretariat and client 
departments to develop business cases that clearly assign 
accountabilities for benefit measurement plans and for tracking and 
reporting of benefits where applicable and justified. 


Most of the large IT projects we audited have taken steps to address capacity issues 


2.86 Organizational capacity refers to the technical and managerial 
ability to deliver an IT project. It also refers to the ability of the entire 
organization to improve the way it does business by using all of a 
system’s capabilities. 


2.87 In 2006, we examined whether, for the seven projects we 
selected, departments and agencies had clearly demonstrated that 
their organization was ready to accept the business transformation 
that came with the project and whether they had the capability and 
commitment to successfully deliver the project. We reported that four 
of the departments lacked the appropriate skills and experience to 
manage the projects or the capacity to use the system to improve the 
way they deliver their programs. 


2.88 The Secretariat committed to develop and implement an 
Organizational Project Management Capacity Assessment Tool and a 
“Project Complexity and Risk Assessment Tool” to assist departments 
to better understand their capacity to manage projects and to better 
ascertain the level of risk and complexity of proposed projects 
(paragraph 2.32). We examined whether the projects included in this 
audit took steps to improve their organizational capacity with, among 
other things, the use of these tools. 


2.89 Expenditure Management Information System (EMIS). 
EMIS was designed to support all aspects of government expenditure 
management by providing information on the cost of meeting the 
government's priorities and achieving particular results. The favoured 
delivery approach was through an in-house system development. 
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2.90 After spending over $34 million on this project since its 
inception in 2000, the Treasury Board of Canada Secretariat realized 
that the project had to be re-scoped to accommodate capacity issues 
and to adopt a more widely applied technology. There was also a need 
to substantially reduce the risks associated with human resource issues, 
both within the Secretariat and from professional services contracts. 
We found that business stakeholders were consulted; however, the 
Secretariat did not prepare an organizational capacity assessment prior 
to revising the project deliverables after 2006. 


2.91 Global Case Management System (GCMS). In 2006, we 
reported that the GCMS'’s organizational capacity was deficient. 

The project had experienced untimely shortages of key resources, and 
Citizenship and Immigration Canada had no previous experience in 
developing and implementing a project of a similar size and complexity. 
Since then, CIC has assessed its capacity to deliver such a project 
through independent reviews and internal audits. Also, it has produced 
plans for business preparation, transition, and operational phases. 
Moreover, instead of attempting to deliver the project as a whole in 
one implementation, the Department adopted a phased approach to 
implementing GCMS. This approach has provided the organization 
with greater capacity to implement and manage the change. 


2.92 Integrated Revenue Collections (IRC). Since our 2006 audit, 
the Canada Revenue Agency has had to address certain organizational 
capacity issues. In 2006, the Agency prepared initial staff resource 
plans and staffing concerns were discussed at senior committees. As a 
result, CRA included in the 2007 business case a new human resource 
strategy concentrating on project challenges related to capacity and 
availability of expert resources. This helped CRA address the resource 
shortfall previously identified. 


2.93 Temporary Resident Biometrics Project. In June 2009, an 
independent review of the biometrics project recommended that the 
project team strengthen its project management expertise. The project 
team hired two directors to address this gap. As well, in response to 
weaknesses identified in the Organizational Project Management 
Capacity Assessment completed in October 2009, the project team 
took steps to define and group the discrete elements for the project, 
to better describe deliverables and identify tasks required to complete 
them. In August and September 2010, the project team took further 
steps to determine whether Citizenship and Immigration Canada had 
the right mix and appropriate number of people to ensure effective 
implementation. The Department has not yet approved the project’s 
human resource management plan. 
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2.94 In summary. We found that for three of the four projects 
included in this portion of the audit, progress on improving 
organizational capacity was satisfactory. EMIS did not asséss 
organizational capacity as required (Exhibit 2.6). 


Exhibit 2.6 Progress in addressing our recommendation on organizational capacity 


At the start of a project, as aaa agencies should cient 
demonstrate that their organization is ready to accept the business 
transformation that comes with the project and has the capability 
and commitment to successfully deliver the project. Specifically, 
departments and agencies should analyze their track record in 


; ‘ Aer : ; Satisfactory 
* completing projects of similar size and complexity; (GCMS, IRC 
* completing skills appraisals and plans to address shortfalls; Biometrics) 
* planning for business preparation, transition, and operational Unsatisfactory 
phases; and (EMIS) 


* considering key stakeholder buy-in and commitment. 


(Recommendation 3.98 of the 2006 November Report of 
the Auditor General of Canada, Chapter 3, Large Information 
Technology Projects) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Most of the large IT projects we audited have taken steps to address project risk 
management issues 


2.95 Sound project management ensures that projects produce 
successful outcomes, within a specified time. A key component of 
project management is effective risk management, which involves 
assessing the potential impact of risks, continuously monitoring the 
risks, and dealing with potential risks before they arise. 


2.96 In 2006, we found that the quality of project risk management 
varied widely from project to project—from good to seriously flawed. 
Although we found that four of the seven projects we audited were 
well managed, the quality of project risk management varied widely. 
Those with weak project management practices experienced long 
delays and large cost overruns. 


2.97 The Secretariat developed a Project Complexity and Risk 
Assessment Tool that includes an assessment of how departments 
manage project risks. None of the projects we examined in this audit 
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were subjected to this tool because departments have until April 2012 
to implement it. 


298 Expenditure Management Information System (EMIS). EMIS 
has successfully completed two of its four phases on time and within 
budget. We observed that project risks had been adequately managed. 
For example, the project’s 2006 Business Case Risk Management Plan 
indicated preliminary identified risks, including, for each risk, probability 
of occurrence, impact, severity, and risk response (risk mitigation). 


2.99 Risk information, such as executive project dashboards, was 
regularly reported to the EMIS Steering Committee. All risks related 
to the first two phases of the project were addressed. 


2.100 Global Case Management System (GCMS). For the current 
phase of GCMS, ending on 31 March 2011, Citizenship and 
Immigration Canada has a risk management approach in place, 
consisting mainly of a risk register capturing major existing, 
anticipated, and emerging risks that are assigned to a specific owner 
and analyzed by likelihood and impact. 


2.101 The risk register contains action plans and status updates and is 
reviewed on a regular basis by the Senior Review Board. However, the 
risks related to the work that remains after this phase is complete have 
not been documented (paragraph 2.49). 


2.102 Integrated Revenue Collections (IRC). We reviewed the risks 
identified in the risk management framework document for the IRC 
project. We selected specific risks initially recorded in both the 2006 
and 2007 business cases, and assessed how these were being managed. 
We found that the Canada Revenue Agency has implemented a formal 
assurance process for assessing the adequacy of its risk management. In 
addition, regular progress reports that include summary information on 
the status of current project risks are submitted to the Resource 
Investment Management Committee for review and feedback. 


2.103 Temporary Resident Biometrics Project. Given that the 
biometrics project is complex and high risk, mechanisms for 
identifying, assigning ownership of, addressing, and communicating 
risks to partners are essential. We found that the biometrics project has 
mechanisms for identifying and assigning ownership of risks, including 
a risk register and an issue-tracking log, and that it communicates 
these risks through its governance structure. 


2.104 However, we found weaknesses in the project’s mechanisms 
for addressing risks and analyzing their impacts in a comprehensive 
manner. For example, we found that, while delivery partners were first 
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made aware of the project’s key risk—delays in the RCMP Real-Time 
Identification (RTID) project—in April 2009, the project team did not 
develop a contingency plan until November 2010. We found that this 
contingency plan provides a general overview of costs and options, but 
does not provide specific details on how the project team would 
implement each option. This gap is problematic, because the timely 
implementation of the RTID project, which was on hold in May 2010, 
is critical to the successful implementation of the biometrics project. 
Without a functioning RTID, Citizenship and Immigration Canada 
will not be in a position to assess criminality to ensure applicants 
posing a risk to the safety and security of Canada are denied entry. 


2.105 We also found that the project team had to redevelop key 
planning documents due to the funding shortfall and changes to scope, 
and is now experiencing delays of about a year as a result. However, 
the project team has not analyzed the impacts of these delays in a 
comprehensive manner. Given these developments, the project team’s 
ability to deliver on the March 2013 implementation date is at risk. 


2.106 In summary. We found that overall progress on the 
implementation of risk management for the projects we reviewed has 
been satisfactory, although progress is unsatisfactory for the Temporary 
Resident Biometrics Project (Exhibit 2.7). 


Exhibit 2.7 Progress in addressing our recommendation on project risk management and reporting 


Recommendation | Progress 


Departments and agencies should 


comply with the Treasury Board Secretariat project 
management guidance, 


* closely manage the project risks, Satisfactory 
* monitor key success factors, and (GCMS, IRC, 
‘ EMIS) 
* regularly report to management any significant internal or 
external event that may prevent the expected project benefits Unsatisfactory 
‘or result in cost overruns or in the project not being delivered (Biometrics) 


on time. 


(Recommendation 3.111 of the 2006 November Report of 
the Auditor General of Canada, Chapter 3, Large Information 
Technology Projects) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 
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Conclusion 


2.107 Overall, the government’s progress on its commitments to our 
2006 recommendations is unsatisfactory (Exhibit 2.8). In the area of 
project governance, two of the four projects we examined (the 
Expenditure Management Information System and the Global Case 
Management System) have had important deliverables deferred 
without full analysis of the impact, costs, and risks of not completing 
these projects. In the area of business cases, three of the five projects 
we examined (the Global Case Management System, Integrated 
Revenue Collections, and Secure Channel) have not quantified 
benefits to be achieved or measured them. 


2.108 As illustrated in Exhibit 2.8, progress has been satisfactory in 
the areas of organizational capacity and project risk management. 

We found that three of the four projects we audited have adequately 
assessed their technical and managerial ability to deliver the project. 
We also found that three of the four projects we audited have properly 
assessed and monitored the potential impacts of risks. 


2.109 The Treasury Board of Canada Secretariat’s work in reviewing 
and challenging large IT projects before they are submitted to Treasury 
Board for approval has improved their likelihood of success. The 
Secretariat's Chief Information Officer Branch has also established 
several monitoring mechanisms to assess the health of the government's 
portfolio of large IT projects since our last audit. However, it is too early 


Exhibit 2.8 Overall progress against our recommendations for the projects 


Expenditure Management 
Information System 


Global Case Management System 


Integrated Revenue Collections 


Secure Channel 


Temporary Resident Biometrics 


Project 


a 


VVCla NGLINE Wy Wh 


Unsatisfactory Satisfactory Unsatisfactory Satisfactory 


Satisfactory Unsatisfactory Satisfactory Satisfactory 
N/A _ | Unsatisfactory N/A N/A 


Unsatisfactory Unsatisfactory Satisfactory Satisfactory 


Satisfactory Satisfactory Satisfactory Unsatisfactory 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, and the time that has elapsed since 


the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, and the time that has elapsed since 


the recommendation was made. 
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to assess the results of the monitoring activity because the Chief 
Information Officer has only recently (September 2010) provided the 
assessment to the Treasury Board ministers for their consideration. 


The Secretariat has responded. While the Treasury Board of Canada 
Secretariat accepts the findings of the audit, the Secretariat is of the 
view that the overall characterization of the government’s progress 

on the management of large IT projects as “unsatisfactory” is not 
warranted. On balance, the Secretariat would find that overall progress 
is indeed “satisfactory,” given the actions taken by the Secretariat related 
to its policy leadership role and the findings related to the individual IT 
projects reviewed. The difference of views on the overall rating rests on 
the weight the Secretariat would accord to the actions presented in this 
report, which the OAG has found as largely “satisfactory.” 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these 
standards as the minimum requirement for our audits, we also draw upon the standards and practices of 
other disciplines. 


Objectives 


The objective of our audit was to determine whether the selected departments and agencies have made 
satisfactory progress in implementing the recommendations related to the November 2006 Report, 
Chapter 3, Large Information Technology Projects. 


Scope and approach 


We examined the Treasury Board of Canada Secretariat’s role in challenging large IT projects and 
supporting Treasury Board ministers’ oversight, as well as its provision of guidance to departments on 
effective management and oversight of their IT projects. 


We also examined four projects that did not meet all of our audit criteria in the 2006 chapter and we 
selected one additional project that was approved since our last audit. The projects selected are with the 
following entities: Canada Revenue Agency, Citizenship and Immigration Canada, Public Works and 
Government Services Canada, and the Treasury Board of Canada Secretariat. 


We also reviewed actions taken in relation to the government’s commitments in response to the 
recommendations the Parliamentary Standing Committee on Public Accounts made in its 7th and 
4th reports to the House of Commons on 25 February 2008 that were relevant to issues within our 
audit scope. 


Criteria 


The Secretariat has an adequate governance framework for Policy on Management of Information Technology, section 6.1.4, 
the government-wide portfolio of large IT investments Treasury Board, 2007 
under development. 


i 
The Secretariat has set a government-wide strategic direction for | * Policy on Management of Information Technology, section 8, 
_ the management of large IT projects. Treasury Board, 2007 


* Directive on Management of Information Technology, 
section 8.1.2, Treasury Board, 2009 


alee. 
The Secretariat has adequate systems and practices in place to Policy on the Management of Projects, sections 1.3 and 5.1, 
support the successful delivery of large IT projects _ Treasury Board, 2009 
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The Secretariat has adequately challenged and monitored 
departmental compliance with its policies, directives, and 
standards regarding investment plans and project submissions 
for large IT projects. 


———— 


Project Management Policy (Monitoring), Treasury Board, 2007 


The Secretariat and, more specifically, the Chief Information 
Officer Branch has in place measures to monitor and report on 
large IT projects. 


Project Management Policy (Monitoring), Treasury Board, 2007 


IT Project Review and Oversight, Treasury Board of Canada 
poy erie ntormiation ie pene 


The selected entities have adequate large IT project governance 
and oversight mechanisms in place that are documented and 
maintained. 


° Policy on the Management of Projects, section 6; f, is : 
Treasury Board, 2009 


¢ COBIT PO10.3 Project Management Approach 


The selected entities have adequately defined the key roles and 
responsibilities of the key personnel responsible for the selected 
large IT projects. 


¢« COBIT PO10.3 Project Management Approach 


Val IT IM5 Develop the detailed candidate project 
business case 


The selected entities have in place adequate procedures for 
monitoring and reporting on the selected large IT projects. 


¢ Policy on the Management of Projects, section 6.2, 
Treasury Board, 2009 


COBIT PO10.13 Project Performance Measurement, 
Reporting and Monitoring 


The selected entities have adequate management plans in place 
that define, measure, and monitor the realization of benefits for 
the large IT projects selected. 


¢ Policy on the Management of Projects, section 6.2, 
Treasury Board, 2009 


¢ Val IT IM4 Develop full life-cycle costs and benefits 


The selected entities have included the key stakeholders of the 
large IT projects in their communication plan. 


COBIT PO6.5 Communication of IT Objectives and Direction 


The Pape entities have asain defined ‘the business goals 


for their large IT project and how the project will deliver on them. 


Tae Case Guide, Easton < ley ities aad 
Treasury Board of Canada Secretariat 


The selected entities have identified and measured the critical 
success factors for each large IT project. 


| Business Case Guide, section 1.1.4, Business Outcomes, 


Treasury Board of Canada Secretariat 


The selected entities have involved the key stakeholders for their 
large IT projects in the decision-making process. 


Business Case Guide, sections 1.4.2: Stakeholder Analysis, 
and 2.2, List of Possible Options, Treasury Board of Canada 
Secretariat 


The selected entities have prepared an adequate options analysis 
case for their large IT project. 


Business Case Guide, section 2, Preliminary Options Analysis, 
Treasury Board of Canada Secretariat 
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Standard for Organizational Project Management Capacity, 
Treasury Board of Canada Secretariat 


Each selected entity has done an adequate assessment of its 
capacity to deliver its large IT project. 


* Standard for Organizational Project Management Capacity, 
Treasury Board of Canada Secretariat 


* COBIT PO10.8 Project Resources 


Each selected entity has adequate human resources available to 
implement its large IT project. 


Each selected entity has an adequate process to identify, assess, | * Integrated Risk Management Framework, Element 3: 
and assign responsibility for and monitor current and emerging | Practising Integrated Risk Management, Treasury Board of 
risks for their large IT projects. | Canada Secretariat 


© COBIT PO10.9 Project Risk Management 


1 


Each selected entity has kept their risk management plans up to. | COBIT PO10.9 Project Risk Management 
date for their large IT projects. 


“Each selected entity has in place an adequate process for | integrated Risk Management Framework, Element 3: Practising 
managing and monitoring progress and changes to requirements | Integrated Risk Management, Treasury Board of Canada 
that is based on a risk analysis. Secretariat 


Each selected entity has an adequate project re-justification at COBIT PO10.6 Project Phase Initiation 
each major checkpoint (stage) for their large IT project that takes 

into account risk-based changes in the business strategy, scope, 
and objectives. 


Each selected entity has a third-party assurance process for COBIT ME4.7 Independent Assurance 
assessing the adequacy of risk management for their large IT 


project. 
et Oe SS SS 2 1 eee SS 


| 
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: 


Management reviewed and accepted the suitability of the criteria used in the audit. 


Period covered by the audit 

The audit covered the period between November 2006 and October 2010. However, some documents 
reviewed go back to 2000. Audit work for this chapter was substantially completed on 29 October 2010. 
Audit team 


Assistant Auditor General: Nancy Y. Cheng 
Principal: Richard Brisebois 
Directors: Bernard Battistin (Lead), Tony Brigandi, Greg Boyd, Marie-Claude La Salle 


Bridget O’Grady 
Jessica L. Perkins 
William Xu 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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Appendix List of recommendations 


The following is a list of recommendations found in Chapter 2. The number in front of the 
recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses 
indicate the paragraphs where the topic is discussed. 


Progress in key management areas 


2.44 The Treasury Board of Canada 
Secretariat should analyze the risks and 
impact of deferring the last two phases 
of the Expenditure Management 
Information System (EMIS) indefinitely 
and take the necessary action 
consistent with the Secretariat’s 


priorities and future objectives for the 
project. (2.38-2.43) 


2.51 Citizenship and Immigration 
Canada should prepare a plan to 
address the costs and risks related to 
the remaining work to be done after 
completion of Release 2 of the Global 
Case Management System (GCMS) in 
order to achieve the project’s overall 
business objectives. (2.45-2.50) 


The Secretariat’s response. Agreed. The Secretariat will 
complete the documentation of the decision to cancel the last 
two phases originally planned for the EMIS project 

by April 2011. The risks and benefits of the decision have been 
assessed, and the decision is consistent with the Secretariat’s 
priorities and objectives. 


The Department’s response. Agreed. Over time, the 
implementation of the GCMS project has been adapted to meet 
the changing business and operational needs of Citizenship and 
Immigration Canada (CIC). The overall project plan was 
amended in 2008 to focus on overseas immigration processing. 
GCMS Release 2 will be successfully delivered worldwide 

by March 2011, fulfilling the scope of the Major Crown Project. 
CIC is preparing a five-year Roadmap to achieve its overall 
business needs, which have evolved since the original business 
objectives of GCMS were established. As we move forward, we 
will continue to build on the GCMS platform and address any 
risks related to the changes to the original scope. The first year 
of this Roadmap will be funded through the CIC Business 
Planning and Investment Planning process, and the next 
deployment is scheduled for summer 2011. As business priorities 
shift and change quickly, the yearly process of approval will be 
followed for GCMS over the coming years. A portion 

of immigration inland processing will be implemented in the 
Case Processing Centre Mississauga and CIC Etobicoke in 
March, with minimal IT changes to the system. 
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2.85 When departments and agencies 
develop a business case, they should 
include a benefit measurement plan 
that 


* contains specific quantifiable benefits 
including clear benchmarks, 


* assigns responsibilities for delivering 
identified benefits, and 


¢ includes a process for measuring and 
reporting benefits. 


(2.59-2.84) 


Chapter 2 


The Canada Revenue Agency’s (CRA’s) response. Agreed. 
The Agency believes that since the end of the audit 

(31 October 2010) it has addressed the recommendation in the 
following manner: 


As noted by the Auditor General, a benefits measurement plan 
for Integrated Revenue Collections (IRC) Phase 1 had been 
completed by the CRA in 2009. The benefits measurement plan 
for IRC Phase | was further refined in 2010 to reflect annual 
gains to be achieved, benchmark information, and a data 
collection strategy, including when benefits would start to be 
realized, and who would be responsible for the delivery of the 
identified benefits. The 2010 revised plan was recently approved 
and work is ongoing in respect of measuring and reporting on the 
benefits. In fact, the approval process in place at the Agency 
requires that a benefits realization plan be prepared as part of 
business cases for large [T-enabled projects, which is an 
established best practice. 


Citizenship and Immigration Canada’s (CIC’s) response. 
Agreed. Future projects will leverage the CIC Performance 
Measurement Framework and Operational Baselining Strategy 
in order to quantify, measure, and report on benefits. 


Public Works and Government Services Canada’s (PWGSC’s) 
response. Agreed. PWGSC is committed to adhering to 
applicable Treasury Board policies, as well as Treasury Board of 
Canada Secretariat Chief Information Officer Branch guidelines 
and standards on the development of business cases. In cases 
where PWGSC is responsible for the implementation of large, 
horizontal IT projects with government-wide or mullti- 
departmental impacts, the Department will continue to work 
cooperatively with the Secretariat and client departments to 
develop business cases that clearly assign accountabilities for 
benefit measurement plans and for tracking and reporting of 
benefits where applicable and justified. 
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What we examined 


Why it’s important 


Internal Audit 


Main Points 


In November 2004, we reported considerable variation among 

six federal organizations in the extent to which their internal audit 
activity met the international standards for the professional practice of 
internal auditing issued by the Institute of Internal Auditors. We also 
reported the extent to which they complied with the Treasury Board 
Policy on Internal Audit. We found that the Treasury Board of Canada 
Secretariat had yet to establish and fund a strategy that would enable it 
to meet the requirements of the Policy and the expectations of the 
internal audit community. We made a number of recommendations 
aimed at improving the quality of internal audit across government. 


Our 2004 report noted that, effective 1 June 2004, the government 
had re-established the Office of the Comptroller General to strengthen 
comptrollership and oversight across the federal government. The 
Comptroller General’s key duties included setting or reviewing internal 
auditing standards and policies of the Government of Canada, 
providing leadership to ensure and enforce appropriate internal 
controls, and promoting sound resource stewardship at all levels across 
the federal government. 


For this status report, we examined the extent of progress made by the 
government in acting upon the commitments it made in response to 
the observations and recommendations of our 2004 report. We looked 
at whether independent audit committees had been established in the 
24 largest departments and agencies that represent about 95 percent of 
the government's total assets, liabilities, revenues, and expenses. We 
assessed the practices and procedures used by a sample of internal 
audit activities. We also looked at whether the Office of the 
Comptroller General provided appropriate oversight and guidance to 
the internal audit activity in departments. 


Audit work for this chapter was substantially completed on 


28 November 2010. 


Internal audit is an important element of good governance. An 
effective internal audit activity can provide senior management with 
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objective, independent assurance that the organization’s financial, 
administrative, and operational controls and management practices 
are effective. It can also suggest improvements that may be needed. 


- The government has made satisfactory progress in acting upon the 


commitments it made in response to the observations and 
recommendations of our 2004 audit. 


* Independent departmental audit committees have been established 


in the 24 large departments; the majority of their members are from 
outside the federal public administration and have the collective 
skills and experience required to provide deputy heads of 
departments with objective advice and recommendations. 
Departmental audit committees are at varying stages in developing 
their practices and procedures. Some committees have been in effect 
for three years and others less than one year. We noted that the 
Canada Revenue Agency established its audit committee in 1999. 
We have also noted the positive impact that established 
departmental audit committees are having in contributing to 
stronger internal audit in government. 


The government has strengthened its internal audit capacity since 
our last audit in 2004. While few internal audit activities in 
departments have undergone an external quality assessment review, 
our own review of a sample of internal audit activities found that 
they generally or partially conformed to policy requirements and to 
the standards of the Institute of Internal Auditors. Internal audit 
reports we reviewed met those standards and requirements. We 
noted that the strong senior management support for internal audit, 
coupled with the impact of departmental audit committees has been 
accompanied by a strengthened internal audit capacity. Senior 
management has indicated that it has a greater appreciation of the 
role that internal audit can and should play within an organization. 


* The Office of the Comptroller General has provided direction and 


guidance to the internal audit and departmental audit committee 
communities through the establishment of a sound policy framework 
and related guidance. We noted, for example, orientation material 
that included roles, responsibilities, and expectations of audit 
committee members. The Office of the Comptroller General has also 
developed tools and guidance for the internal audit community, such 
as risk-based audit plans and guidance on core controls. 


The entities have responded. The entities agree with our 
recommendation. Their detailed responses follow the recommendation 
in the chapter. 
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Internal audit activity—A department, 
division, team of consultants, or other 
practitioner(s) that provides independent, 
objective assurance and consulting services 
designed to add value and improve an 
organization's operations. The internal audit 
activity helps an organization accomplish its 
objectives by bringing a systematic, disciplined 
approach to evaluate and improve the 
effectiveness of governance, risk management, 
and control processes. 


Independence—The freedom from conditions 
that threaten objectivity or the appearance of 
objectivity. Such threats to objectivity must be 
managed at the individual auditor, engagement, 
functional, and organizational levels. 


Objectivity—An unbiased mental attitude that 
allows internal auditors to perform engagements 
in such a manner that they have an honest belief 
in their work product and that no significant 
quality compromises are made. Objectivity 
requires internal auditors not to subordinate 
their judgment on audit matters to others. 


Internal auditing—An independent, objective 
assurance and consulting activity designed to 
add value and improve an organization's 
operations. Internal auditing helps an 
organization accomplish its objectives by 
bringing a systematic, disciplined approach to 
evaluating and improving the effectiveness of 
risk management, control, and governance 
processes. 


INTERNAL AUDIT 


Introduction 


31 An effective internal audit activity is a fundamental component 
of strong governance. It can provide senior management and audit 
committees with assurances that key financial, administrative, and 
operational activities, and the organization’s management practices, 
are efficient and effective. An internal audit can also suggest ways to 
improve these activities and practices. 


3.2 Internal audit is one of several ways that an organization may 
assess and oversee management practices and find out whether the 
organization is achieving its objectives. Other ways include program 
evaluation, studies, and monitoring whether the organization's 
practices are adequate and effective. 


3.3. The attributes of independence and objectivity distinguish 
internal auditing from other activities within an organization that 
review internal practices. 


3.4 The Treasury Board Policy on Internal Audit (the Policy) gives 
the guidelines for internal audit activities that departments are to 
follow. Roles and responsibilities for implementing the Policy are 
summarized in Exhibit 3.1. 


3.5 The Canada Revenue Agency, which was included in this 
audit, is structured differently than the other entities. The Agency 
was established as a separate agency on | November 1999. Enabling 
legislation provided for the establishment of a Board of Management 
to oversee the organization and administration of the Agency. The 
Board comprises external members who are independent of the 
Agency. The Board’s structure includes an audit committee whose 
mandate is to provide oversight of the internal audit activity. The 
Canada Revenue Agency Act provides the legislative authority for the 
Internal Audit Policy of the Agency, which is approved by the Board 
of Management. While not subject to the requirements of Treasury 
Board policies, the Agency aims to respect their intent. 
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Exhibit 3.1 Roles and responsibilities for implementing the Policy on Internal Audit 


Deputy heads Establish an internal audit activity that is appropriately 
resourced and that operates in keeping with the Policy 
and with the Institute of Internal Auditors (IIA) 


Standards. 


Form an independent departmental audit committee, 
where a majority of members are from outside the 
federal public administration. 


Appoint a qualified chief audit executive at a senior 
executive level to lead and direct the internal audit 
activity. The chief audit executive reports directly to 
the deputy head. 


Approve a departmental internal audit plan that 
addresses all areas of higher risk and significance, 
including audits the Comptroller General has identified 
as part of government-wide or sectoral coverage. This 
plan is to be designed to support an annual opinion 
from the chief audit executive on departmental risk 
management, control, and governance processes. 


Ensure appropriate internal audit coverage for special 
operating agencies and other entities within their 
departments and under their control. 


Departmental audit As part of their mandate, committees review and report on 

committees those areas of responsibility that the deputy head has 
chosen in that year’s risk-guided focus. The overall areas of 
responsibility are 


to review management’s arrangements to promote 
public service values and ethics and to ensure that the 
department complies with laws, regulations, policies, 
and standards of conduct; 


to review the corporate risk profile and the department’s 
risk management arrangements; 


to assess the internal audit activity; 


to review the work of the Office of the Auditor General 
and other agents of Parliament and central agencies; 


to review follow-up on management action plans; 


to review financial statements and Public Accounts of 
Canada reporting; and 


to review risk and accountability reporting. 
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Exhibit 3.1 Roles and responsibilities for implementing the Policy on Internal Audit (continued) 


Office of the 
Comptroller General 


Chief audit executives 


Provide annual opinions to deputy heads and 
departmental audit committees on the effectiveness and 
adequacy of risk management, control, and governance 
processes in their departments, and report on individual 
risk-based audits. 


Create appropriate policies and procedures to guide the 
internal audit activity. : 


Create a risk-based audit plan. 


Ensure that the internal audit activity has the number of 
qualified staff it needs to carry out the risk-based audit 
plan, in keeping with the IIA Standards. 


Ensure that the work of the internal audit activity is 
carried out in keeping with the IIA Standards. 


Ensure that the departmental audit committee receives 
the information it needs to carry out its responsibilities. 


Is responsible for horizontal and sectoral audits in large 
and small departments and agencies; 


Is responsible for focused, sustained functional 
leadership of internal audit across government to build 
and develop capacity; to ensure adequate levels of 
professionally qualified staff; and to ensure that 
departments maintain professional standards and 
rigorous methodology in delivering internal audits; 


Reports periodically to Treasury Board on the state of 
risk management, control, and governance processes 
across government; these reports address fundamental 
controls, including basic reporting controls for financial 
statements, thematic or sectoral controls, and the 
results of risk-based internal audit work done within 
departments; 


Provides strategic direction, leadership, advice, and 
assistance on issues related to internal audit; 


Develops and updates relevant policies, directives, 
guidelines, and standards for internal audit; 


Provides policy guidance and interpretation to 
departments and agencies on how to implement relevant 
policies, directives, guidelines, and standards; and 


Monitors how relevant policies are implemented and 
whether policy objectives are being achieved. 


Sources: Treasury Board Policy on Internal Audit; Treasury Board Directive on Departmental Audit 
Committees; Treasury Board Guidelines on the Expected Qualifications for Chief Audit 
Executives; Treasury Board Guidelines on the Responsibilities of Chief Audit Executives; and the 
Institute of Internal Auditors International Professional Practices Framework 
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What we found in 2004 


3.6 We last carried out a government-wide assessment of internal 
audit in 2004. We found wide variations in how well the internal audit 
activity in six federal organizations had met the Institute of Internal 
Auditors International Standards for the Professional Practice of 
Internal Auditing (the ITA Standards). We also found variations in 
how well they complied with the Treasury Board Policy on Internal 
Audit that was in effect at that time: 


* In two organizations (Public Works and Government Services 
Canada and the Royal Canadian Mounted Police), the internal 
audit activity generally met the IIA Standards. 


¢ Three departments (Foreign Affairs and International Trade 
Canada, Human Resources and Skills Development Canada, and 
Natural Resources Canada) partially met the IIA Standards. 


* One agency (Canadian International Development Agency) did 
not meet many of the IIA Standards. 


3.7 Our report identified a number of important factors that, if 
implemented, could have a positive influence on the quality of internal 
audit across government: 


* aconsistent understanding by senior management of the role that 
internal audit can and should play; 


* a departmental audit committee with external members who are 
independent of management; 


¢ aclear human resource strategy for departments and central 
agencies that sets out the qualifications and appropriate number 
of staff for the internal audit community; 


* a focus on assurance services; and 


* a strategy to ensure appropriate internal audit coverage and 
capacity in small entities. 


3.8 We also found that the Treasury Board of Canada Secretariat 
(the Secretariat) had yet to create and fund a strategy that would 
enable it to meet both the requirements of the Policy on Internal Audit 
that was in effect and the internal audit community’s expectations. 
The Secretariat agreed with these observations and agreed to take 
action to correct the weaknesses we noted. 
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Events since 2004 


3.9 ‘To respond to these issues and to strengthen the government’s 
internal audit capacity, the Treasury Board adopted a revised Policy on 
Internal Audit (the Policy) in April 2006. The revised Policy included 
related directives, guidelines, and standards. The Policy and its related 
guidance were revised again in July 2009. 


3.10 The revised Policy and related guidance were designed to sustain 
a strong, credible internal audit activity that 


* has the confidence of the government; 


* contributes directly to sound risk management, control, and 
governance; and 


* is well-positioned to support governance within departments and 
agencies across government. 


The Policy requires that independent departmental audit committees be 
created and sets out specific requirements for internal audit activities, 
deputy heads, departmental audit committees, and the Comptroller 
General of Canada. Internal audit activities are to follow the Policy and 
the ITA International Professional Practices Framework (including the 
IIA Standards), which was adopted by Treasury Board in its Policy. 


3.11 The internal audit community is also expected to respond to the 
requirements of the Financial Administration Act, which calls for deputy 
heads-to ensure that the department has an internal audit capacity. 


Focus of the audit 


3.12 The focus of our audit was to find out whether departments, 
agencies, and the Office of the Comptroller General of Canada have 
acted upon the commitments they made in response to the 
observations and recommendations of our 2004 report. 


3.13. The audit examined whether departments and agencies had set 
up independent audit committees, as the Policy requires, and whether 
the committees were carrying out the responsibilities that the Policy 
and related directives and guidelines set for them. 


3.14. We looked at the audit committees of the 24 large departments 
and agencies that represent approximately 95 percent of the 
Government of Canada’s total assets, liabilities, revenues, and expenses. 
(A list of these departments and agencies is found in About the Audit.) 


3.15 The audit also assessed whether the internal audit activity met the 
requirements of the I[A Standards, which Treasury Board adopted in its 
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Policy. We examined four key attributes of an effective internal audit 
activity in a random sample of 12 of the 24 large departments. (A list of 
the 12 departments in the random sample is found in About the 
Audit.) 


3.16 Specifically, we assessed the following aspects: 
* the reporting relationship of internal audit; 
¢ the risk-based audit plan that the internal audit activity created; 


* whether the reporting practices met the requirements of the 


Policy and the IIA Standards; and 


* whether the internal audit activity had completed an external 
quality assessment review, as the IIA Standards require. 


3.17 We also conducted quality assessment reviews (Exhibit 3.2) in 
six of the large departments. (A list of the six departments that 
received such a review is found in About the Audit.) 


3.18 The final area the audit addressed was whether the Office of the 
Comptroller General (the OCG) was providing leadership and direction 
for the internal audit community and for departmental audit committees. 


3.19 In our audit, we examined the practices and procedures that 
departmental audit committees and internal audit have put in place 
over the last three years. 


3.20 More details on the audit objectives, scope, approach, and 
criteria are in About the Audit at the end of this chapter. 


Exhibit 3.2 Purpose of a quality assessment review 


A quality assessment review is an external assessment that includes assessing the 
following elements of the internal audit activity: 


conformity with the definition of internal auditing, the IIA Code of Ethics, and the 
IIA Standards, and with the internal audit activity’s charter, plans, policies, 
procedures, practices, and legislative and regulatory requirements that apply; 


the board’s, senior management's, and the operational managers’ expectations; 


the integration of the internal audit activity into the organization’s governance 
process, including the relationships between and among the key groups involved in 
the process; 


the tools and techniques used; 


the mix of knowledge, experience, and disciplines within the staff, including staff 
focus on process improvement; and 


a review of whether the internal audit activity adds value and improves the 
organization's operations. — 


Source: Adapted from the Institute of Internal Auditors International Professional Practices Framework 
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Observations and Recommendation 


3.21 Independent departmental audit committees are an important 
part of the governance structure of departments. These committees are 
designed to offer objective advice and recommendations to the deputy 
head on whether the department’s risk management, control, and 
governance frameworks and processes (including accountability and 
auditing systems) are adequate, and on how well they have been 
implemented. 


3.22 In its response to our 2004 audit, the Treasury Board revised its 
Policy on Internal Audit (the Policy) in 2006 toimplement our | 
recommendation that external members be required on departmental 
audit committees. 


3.23 A key element of the Policy was that by 30 September 2009, 
deputy heads must have formed a departmental audit committee, with a 
majority of members from outside of the federal public administration. 
The collective skills, knowledge, and experience of members were to 
allow the committee to carry out its duties competently. Members of the 
departmental audit committee are to be chosen jointly by the respective 
deputy head and the Comptroller General and approved by the Treasury 
Board. Members of the committee who are from within the federal 
public administration must be at the deputy head or associate deputy 
head level. The Canada Revenue Agency appoints its audit committee 
from among members of its Board of Management all of whom are from 
outside the federal public administration. 


3.24 Our assessment on the progress on the following topics is found 
at the end of this section (see page 14). 
Independent departmental audit committees have been established 


3.25 We looked at whether the 24 large departments had established 
independent departmental audit committees in accordance with the 
Policy and related guidelines, which required that independent audit 
committees be in place by 30 September 2009. 


3.26 We noted that, in keeping with the Policy, 23 departmental audit 
committees were formed over the last three fiscal years: 


* ten committees in 2007-08 
* eight committees in 2008-09 


¢ five committees in 2009-10 
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3.21 The Canada Revenue Agency implemented its audit committee 
in 1999, at the time the Agency was established as a separate agency. 


3.28 We reviewed the composition of the audit committees in each of 
the 24 large departments to find out whether the makeup of each 
committee met the Policy’s requirements. Specifically, we assessed the 
date the committee was formed, whether the majority of members were 
from outside the federal public administration, and, in the case of 
department members, whether they were deputy heads or associate 
deputy heads. 


3.29 In each case, members appointed were independent of the 
federal public administration. Each committee had at least one 
member with the financial expertise that the Policy requires. 
Department members of the audit committee were deputy heads 
or associate deputy heads, as the Policy requires. 


3.30 As part of our audit, we reviewed the process for identifying 
and recruiting external audit committee members. We found that the 
Office of the Comptroller General (the OCG) followed a rigorous 
process, using the services of a professional recruitment firm. This 
process identified independent, qualified individuals. Both the 
Comptroller General and the deputy head of the department 
recommended to the Treasury Board that these individuals be 
appointed. We reviewed the background documentation on each 
appointment and concluded that the process led to forming audit 
committees whose members meet the competency profiles that the 
Policy outlines. The Canada Revenue Agency appoints its audit 
committee through a resolution of the Board each fiscal year. All 
Canada Revenue Agency committee members are independent of 
the Agency. 


3.31 The audit committee members and senior department officials 
we interviewed as part of this audit believe that the process was 
effective in identifying qualified individuals. However, they noted that 
the appointment process was lengthy. 


3.32. We found that committee members for individual departments 
were often appointed on the same day, with the same terms of 
appointment. The OCG and departments have said that in renewing 
appointments to audit committees they will use terms of varying 
duration to ensure continuity. 
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Audit committee charters were created in keeping with the Policy on Internal Audit 


3.33 In each of the 24 large departments, we looked to see whether 
the departmental audit committees had created and adopted a charter 
that sets out the committee’s responsibilities, and whether these 
responsibilities matched the Policy’s requirements. We found that in 
each case, the committee had created and approved such a charter. 
We found that the charters were consistent with the guidance 
provided by the OCG. We noted that the Canada Revenue Agency 
audit committee charter also sets out roles and responsibilities for its 
audit committee. These are consistent with the intent of the Treasury 
Board Policy on Internal Audit. 


Audit committee members received appropriate training 


3.34 As part of our audit, we reviewed the training and orientation 
material that departmental audit committee members received. 
Looking at both the type and the amount of training provided, we 
noted that the OCG and individual departments gave audit committee 
members a range of training. 


3.35 We saw that the OCG offered training and development 


programs on the following topics: 


* pillars and perspectives on the role of audit committees in the 


federal public service; 
* financial management and literacy in the federal public service; 


* risk management, frameworks and practices, and values and 


ethics; and 
* management control frameworks. 


The OCG also held departmental audit committee symposiums that 
dealt with a variety of topics. 


3.36 We found that the vast majority of committee members took part 
in the orientation training that the OCG gave. As part of our audit, 
we interviewed a selection of audit committee members to collect 
their views on the training they received. Almost all the members we 
interviewed said that the training was very important to ensuring that 
they understood and carried out the duties expected of them. Members 
whose background was not in government said they found the 
orientation training valuable. Committee members added that this 
training should be continued. 
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3.37 As part of our audit, we reviewed the information and 
orientation that departmental audit committee members received 
about the business of their respective departments. We also asked a 
selection of the members how satisfied they were with the nature and 
amount of orientation material they received. 


3.38 The committee members we interviewed said they were satisfied 
with their department’s efforts to provide orientation on its objectives 
and programs. This material included presentations and key 
performance and accountability documents. Many members also went 
on site visits to see operations first-hand. Audit committee members 
said they felt that the departments had been very responsive when 
asked to give additional material or explanations. 


Audit committees are at varying stages of development 


3.39 Departmental audit committees were designed to provide advice 
to the deputy head. To do so, the Treasury Board Directive on 
Departmental Audit Committees requires the committees to exercise 
oversight of core areas of departmental management, control, and 
accountability, including reporting on the internal audit activity, and 
to provide advice to deputy heads. 


3.40 The Policy requires audit committees to give advice and 
recommendations to the deputy head on the quality and results of 
assurance projects and the adequacy and functioning of the 
department’s risk management, control, and governance frameworks 
and processes (including accountability and auditing systems). 


3.41 As part of their responsibilities, departmental audit committees 
are to review and report at least once a year on these topics: 


* management's arrangements to promote public service values 
and ethics and to ensure that departments comply with laws, 
regulations, policies, and standards of conduct; 


* the corporate risk profile and the department’s arrangements 
for managing risk; 


* the department’s arrangements for internal control; and 


* assessments of the internal audit activities. 


3.42 We found that each of the audit committees in the 24 large 
departments had created work schedules that would allow the 
committees to address all areas of their mandate each year. 
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3.43 As noted earlier, some audit committees were not formed until 
the 2009-10 fiscal year. In some cases, the committees had met only 
once or twice before our audit was completed. While the work 
programs for these committees addressed all required areas of the 
Policy, the committees had not yet had the opportunity to complete a 
full annual cycle. For this reason, these committees had not addressed 
all areas of responsibility. Those audit committees formed in previous 
fiscal years had looked at all areas of responsibility. 


3.44 ‘To find out whether departmental audit committee members 
received appropriate information, we reviewed the material that 
committee members received on the areas of responsibility that the 
Policy describes. We also interviewed a sample of audit committee 
members and deputy heads to see whether they were satisfied with the 
nature and amount of information they received. 


3.45 The audit committee members we interviewed told us they were 
satisfied with the nature and amount of information they received. 
They also said they found that departments responded well to requests 
for more information or details. We noted that the departmental audit 
committees received the information they needed to carry out their 
duties. For example, in the area of risk management, we found that 
departmental officials typically provided committee members copies of 
the corporate risk profiles. In many cases, members also attended 
presentations by assistant deputy ministers on key priorities and risks 
tied to their areas of responsibility. 


3.46 Proper documentation of the discussions and activities of the 
audit committees is an important element of showing the work that is 
being done. As part of our audit, we reviewed the minutes of the 
departmental audit committee meetings. In most cases, the minutes 
contained a recommendation for the deputy head to approve key items 
such as the risk-based internal audit plan or completed internal audit 
reports. In some cases, the minutes said only that the agenda item had 
been discussed. We also noted that the minutes did not always describe 
the type or length of the discussions. For this reason, it was not possible 
to assess the nature and extent of the discussion of key items. 


3.47 However, in interviews we held as part of our audit, audit 
committee members and department managers both said they were 
satisfied with the nature and extent of discussion at audit committee 
meetings. The deputy heads we interviewed said they were satisfied 
with the advice the committees offered. 
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Independent departmental audit committees have had a positive effect 


3.48 We wanted to find out in our audit what effect, if any, the 
departmental audit committees have had on departmental 
management and the internal audit activity. 


3.49 To answer this question, we interviewed a broad range of 
department managers, deputy heads, and assistant deputy heads. 
We also interviewed internal audit management and staff. 


3.50 We found that internal auditors and senior management alike 
said that creating independent departmental audit committees has 
strengthened the independence of internal audit. The chief audit 
executives we interviewed said that these committees significantly 
enabled the internal audit capacity to develop in departments. These 
executives also noted that the active participation of deputy heads 
on the committees served as an important signal within the 
organization that internal audit is a key part of the department's 


governance processes. 


3.51 The deputy heads we interviewed said that the independent 
departmental audit committees brought an increased rigour and 
challenge to reviewing the results and quality of the work of internal 
audit. Deputy heads also noted the important role that 

audit committees play in challenging the completeness of action plans 
that management prepared to respond to completed audits and in 
monitoring progress against the action plans. Exhibit 3.3 indicates our 
assessment of progress in addressing our recommendations on 
departmental audit committees. 


Exhibit 3.3 Progress in addressing our recommendations on departmental audit 
committees 


The Treasury Board Secretariat should establish in the Treasury 
Board’s Policy on Internal Audit a requirement for external 
membership on departmental audit committees. 


(Recommendation 1.40 of the 2004 November Report of 
the Auditor General of Canada, Chapter 1, Internal Audit in 
Departments and Agencies) 


Satisfactory 


The Treasury Board Secretariat should, as necessary, provide 
guidance on better practices for audit committee performance and 
provide guidance on appropriate training for departmental audit 


committee members. Satisfactory 


(Recommendation 1.48 of the 2004 November Report of 
the Auditor General of Canada, Chapter 1) 
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Exhibit 3.3 Progress in addressing our recommendations on departmental audit 
committees (continued) 


The Treasury Board Secretariat should update the roles and 
responsibilities of the audit committee and its members. 
Satisfactory 
(Recommendation 1.49 of the 2004 November Report of 
the Auditor General of Canada, Chapter 1) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


3.52 Internal auditing is intended to be an independent, objective 
assurance and consulting activity that is designed to add value to and 
improve an organization’s operations. It is intended to help an 
organization meet its objectives by bringing a systematic, disciplined 
approach to evaluating risk management, control, and governance 


processes and making them more effective. 


3.53. In arandom sample of 12 of the 24 large departments, we looked 
at the following four aspects of an internal audit activity: 


* reporting relationships, 

¢ risk-based plans, 

* reporting practices, and 

* completion of an independent external quality assessment review. 


3.54 Our assessment of the progress on the following topics can be 
found at the end of this section (see page 21). 


Reporting relationships support the independence of internal audit 


3.55 We looked at whether each of the 12 departments in our sample 
had set up appropriate reporting relationships for the internal audit 
activity. 


3.56 Treasury Board’s Policy on Internal Audit (the Policy) requires 
that the internal audit activity report directly to the deputy head. Such 
a reporting relationship is an essential element in creating and 
preserving the internal audit activity’s independence. 


3.57 We found that in each of the 12 departments, the internal audit 
activity reported directly to the deputy head. We also found that the 
relationship established between the internal audit activity and the 
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departmental audit committees further supported the independence of 
the internal audit activity. 


Annual audit plans focus on key risks 


3.58 We looked at whether each of the 12 departments in our sample 
had developed a risk-based audit plan for the internal audit activity. 


3.59 Government policy and the Institute of Internal Auditors 
International Standards for the Professional Practice of Internal 
Auditing (the IIA Standards) require that the internal audit activity 
develop a risk-based audit plan to set its audit priorities, in keeping 
with the organization’s goals. We noted that the OCG’s Internal Audit 
Sector had developed guidance for departments to use to create a risk- 
based internal audit plan. 


3.60 We found that in each of the 12 departments, the internal audit 
activity had developed a risk-based audit plan in keeping with the 
guidance. We noted that these plans clearly described those areas of 
departmental activities that would be subject to audit in the three-year 
planning horizon and explained why some areas would not be audited. 
In each case, the departmental audit committee had reviewed the risk- 
based audit plan. We found that there is an alignment between risk- 
based audit plans developed by internal audit and the department’s 
risk profiles. 


3.61 We noted that internal audit activities reported regularly to their 
respective audit committees on the status of work completed and key 
changes to their internal audit plan. 


3.62 In October 2010, we reported on the results of our audit of 
expenditures under the government’s Economic Action Plan (the Plan), 
which was introduced in the 2009 budget. That audit looked at whether 
internal audit had considered the changes, if any, in departmental risk 
profiles as a result of Plan expenditures and had adapted the planned 
audit coverage if necessary. Our audit of the Economic Action Plan in 
fall 2010 found that internal audit activities responded to the changes 
in the departmental risk profiles as a result of the Plan. 


The results of internal audit work need to be reported clearly 


3.63 We assessed the reporting practices that our random sample of 
12 departments used to communicate the results of the work that the 
internal audit activities completed. 
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3.64 The IIA Standards require that an internal audit report contain, 
at a minimum, the purpose, scope, and results of the engagement. The 
report is also to include observations, conclusions, opinions, 
recommendations, and management’s action plans to-correct any 
weaknesses noted. 


3.65 In addition, internal audit reports must clearly set out the results 
of the audit work. Reports are to note the extent to which observed 
conditions met expected criteria and, if necessary, the cause and effect 
of any changes in expected conditions. 


3.66 Internal audit reports must meet these requirements because 
doing so clearly explains the nature and extent of the observations so 
that the reader can understand their importance. Also, this 
information helps to explain the impact that observed weaknesses may 
have on the department's ability to achieve its stated strategic 
outcomes or objectives. 


3.67 We saw good practices in some internal audit activities and 
opportunities for further improvement in others. In particular, we 
noted a good practice that Canadian Heritage’s internal audit 
activity adopted: using a rating system to identify whether controls 
are adequate. This approach sums up the department’s performance 
for senior management and the audit committee. The conclusions 
and ranking reached for each of the examination criteria used in the 
audit were developed according to the definitions found in 

Exhibit 3.4. This approach clearly shows relative strengths and 
weaknesses in management practices and focuses on areas that need 
immediate attention. 


3.68 In contrast, we also noted that a number of departments had 
adopted a reporting style that made it difficult for the reader to 
understand the seriousness of observations or the overall significance 
of the findings. In these cases, departments used wording that stated 
that elements of a management control framework were in place, but 
opportunities for improvement remained. 


3.69 Many of the audit committee members and senior managers we 
interviewed as part of our audit noted the need for internal audit 
reports to be more succinct and to focus their recommendations at a 
strategic level. 
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Exhibit 3.4 Canadian Heritage uses a rating system to identify whether controls are adequate 


Is well-managed—with no material weaknesses 
noted—and is effective. 


Is well-controlled 


Is controlled Is well-managed—but minor improvements 


are needed—and is effective. 


Has moderate issues that call for management 
focus (at least one of the following two criteria 
must be met): 


Has moderate 
issues 


* control weaknesses exist, but exposure is 
limited because likelihood of risk occurring 
is not high; or 


* control weaknesses exist, but exposure is 
~ limited because impact of the risk is not high. 


Significant 
improvements 
are needed 


Requires significant improvements (at least one 
of the following three criteria needs to be met): 


* financial adjustments that are material 
to line item or area or to the department 
are required; or 


* control deficiencies represent serious 
exposure; or 


* major deficiencies are found in the overall 
control structure. 


Source: Adapted from Canadian Heritage rating system 


Few departments have had an external quality assessment review 


3.70 The Policy requires each internal audit activity to have an 
external review conducted at least once every five years. 


3.71 Quality assessment reviews cover the entire spectrum of internal 
audit work that the internal audit activity performs. These reviews 
assess internal audit activities using three categories: “generally 
conforms,” “partially conforms,” or “does not conform” to the IIA 
Standards. The ratings are defined in Exhibit 3.5. 


3.72. We looked at whether the 12 departments in our random sample 
had completed a quality assessment review. 


3.73 We found that 3 of the 12 departments—the Canada Revenue 
Agency, Environment Canada, and Canadian Heritage—have had a 
quality assessment review completed. In each case, departments “generally 
conformed” to requirements of the Policy and the IIA Standards. 
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3.74 We also carried out detailed quality assessment reviews on a 
judgementally selected sample of six departmental internal audit 
activities to find out whether they met the requirements of the Policy 
and the ITA Standards. The results of these reviews are shown in 
Exhibit 3.6. Of these six departments, two—National Defence and 
the Canadian International Development Agency—had had an 
external quality assessment review in 2005 and 2004 respectively. 


7 


Exhibit 3.5 Quality assessment reviews use three categories to assess conformity to the IIA Standards 


Generally conforms to the Standards—The relevant structures, policies, and 
procedures of the internal audit activity, as well as the processes by which they are 
applied, complied with the requirements of the Standards. While there may be 
opportunities for improvement, these did not represent situations where the internal 
audit activity had not implemented the Standards, did not apply them effectively, or did 
not achieve their stated objectives. 


Partially conforms to the Standards—The internal audit activity has fallen short of 
achieving some of its major objectives. These will usually represent some significant 
opportunities for improvement in effectively applying the Standards and/or achieving 
their objectives. Some of the deficiencies may be beyond the control of the internal 
audit activity and may result in recommendations to senior management or the board 
of the organization. 


Does not conform to the Standards—tThe internal audit activity is not achieving many 
of the objectives of the Standards. These deficiencies will usually have a significant 
negative impact on the internal audit activity’s effectiveness. They may also represent 
significant opportunities for improvement, including actions by senior management or 
the board. 


Source: Adapted from Institute of Internal Auditors Quality Assessment Manual 


Exhibit 3.6 Most of the departments we selected met the requirements of the Policy on Internal 
Audit and the IIA Standards 


Generally — 
conforms to Partially conforms | Does not conform 
the Policy and to the Policy and to the Policy and 
i Organization the IIA Standards | the llA Standards | the IIA Standards 
Canadian International e 
Development Agency 
Foreign Affairs and ® 
International Trade Canada 
Health Canada © 
National Defence . ) 
Royal Canadian Mounted e 
Police 
Veterans Affairs Canada 0) 
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3.75 In conducting our assessment, we noted that the Royal Canadian 
Mounted Police’s internal audit activity had experienced a significant 
turnover, which meant recruiting and rebuilding the activity. This 
situation had an impact on the assessment. 


3.76 Strengths noted. In conducting these quality assessment 
reviews, we noted a number of strengths that were common to the six 
internal audit activities: 


* an internal audit charter that set out the purpose and mandate of 
the internal audit activity; 


* the departmental audit committee and deputy head had approved 
the internal audit charter; and 


¢ the internal audit charters also set out the authority of the 
internal audit activity in terms of its right of access to the 
department’s books, records, and staff. 


3.77 We also noted that, in each case, the internal audit activity had 
appropriate reporting relationships. It reported directly to the deputy 
head, with oversight provided by an audit committee, the majority of 
whose members were from outside the federal public administration. 
This relationship is important because it supports the independence 
and objectivity of the internal audit activity. We also found that the 
planning activities that were done were well-documented and 
considered key aspects of the area to be audited. 


3.78 Opportunities for improvement. Quality assurance and 
improvement programs for the internal audit activity need to be 
strengthened. Most of the internal audit activities that we reviewed 
were monitoring internal audits already in process. However, at the 
time of our audit, periodic assessments of internal audit activities as a 
whole had not been conducted. Both the monitoring and periodic 
assessment elements are necessary for an effective quality assurance 
and improvement program. 


3.79 In completing the quality assessment reviews, we noted some 
common areas where departments could strengthen their practices. 
As stated in the previous section, the most frequent comment from 
key stakeholders—audit committee members and senior 
management—was about communication and reporting. In internal 
audit reports, these stakeholders were seeking concise observations 
that clearly communicated the strengths and weaknesses noted in 
the audit. 
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Exhibit 3.7 Progress in addressing our recommendation on external quality assessments 


Departments should ensure that their internal audit groups 
conduct an external quality assessment by 1 January 2007. 


(Recommendation 1.92 of the 2004 November Report of Unsatisfactory 

the Auditor General of Canada, Chapter 1, Internal Audit in 

Departments and Agencies) 
: 4 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


3.80 Recommendation. Departments that have not had an external 
quality assessment should have an assessment conducted as required 
by the International Standards for the Professional Practice of Internal 
Auditing and the Treasury Board Policy on Internal Audit. 


The Treasury Board of Canada Secretariat’s response. Agreed. The 
Secretariat’s Internal Audit and Evaluation Bureau ([AEB) has 
planned for the conduct of an external quality assessment for the 
Secretariat as a department. Specifically, a self-assessment exercise was 
conducted in the 2010-11 fiscal year, in preparation for an external 
quality assessment to be completed by fall 2011. 


The Canada Border Services Agency’s response. Agreed. The 
Canada Border Services Agency intends to complete an external 
quality assessment by June 2012, as required by the International 
Standards for the Professional Practice of Internal Auditing and the 
Treasury Board Policy on Internal Audit. 


Canadian Heritage’s response. Agreed. Canadian Heritage notes the 
Office of the Auditor General’s recognition of quality assurance work 
conducted at the Department and will endeavour to undertake an 
external quality assessment every five years. 


Correctional Service Canada’s response. Agreed. We understand the 
necessity and usefulness of quality assessments. As such, we had a 
preliminary external assessment conducted in 2006, where 
recommendations for improvement were made and actions have been 
taken to address them. The results of the assessment and related 
actions have been presented to and approved by the audit committee 
and reported annually in the Chief Audit Executive reports. At this 
point in time, following Treasury Board’s direction, Correctional 
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Service Canada fully expects to have an external inspection conducted 
by the end of the 2011-12 fiscal year. 


The Department of Finance Canada’s response. Agreed. Internal 
audit and evaluation at the Department of Finance is currently 
planning to have an external quality assessment conducted, as 
required by the International Standards for Professional Practice of 
Internal Auditing and the Treasury Board Policy on Internal Audit, by 
September 2012. 


Human Resources and Skills Development Canada’s response. 
Agreed. Corrective measures are well under way to address this 
observation. Prior to initiating a formal external quality assessment 
review in November 2010, the Internal Audit Services Branch 
conducted yearly self-assessments against the Institute of Internal 
Auditors Standards, the Office of the Comptroller General’s Internal 
Audit Self-Diagnostic Tool, and relevant Treasury Board policies. The 
Branch also engaged an independent external reviewer in the 2008-09 
fiscal year to conduct a preliminary assessment of the Department's 
readiness for the formal external assessment in the 2010-11 fiscal year. 
Building on the results of the preliminary assessment, the Department 
developed an action plan to address the gaps identified. The external 
quality assessment review is under way and will be completed 


by 30 April 2011. 


Industry Canada’s response. Agreed. An external quality assessment 
is currently planned for the 2013—14 fiscal year. 


The Department of Justice Canada’s response. Agreed. The 
Department of Justice is planning to conduct an external quality 
assessment during the 2011-12 fiscal year. 


3.81 In response to our 2004 audit, the Office of the Comptroller 
General (the OCG) made a commitment to strengthen the internal 
audit activity in government. The initial step taken was to revise the 
Policy on Internal Audit (the Policy) in April 2006, with further 
refinements in 2009. Key elements incorporated into the Policy to 
strengthen internal audit were 


* professionalizing internal audit activity; 


¢ adopting the Institute of Internal Auditors (IIA) International 
Professional Practices Framework; and 


* creating departmental audit committees, the majority of whose 
members were from outside the public administration, to 
strengthen the independence of the internal audit activity. 
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3.82 The Policy’s implementation was supported by strategic direction 
from the OCG and by supplementary funding to departments to help 
them achieve the Policy’s objectives. 


3.83 The Policy sets out a broad range of responsibilities for the OCG. 
The OCG is required to 


* provide strategic direction, leadership, advice, and assistance on 
issues related to internal audit, and provide policy guidance and 
interpretation to departments and agencies on how to implement 
relevant policies, directives, guidelines, and standards; 


* monitor how relevant policies are implemented and whether 
policy objectives are being achieved, as well as to ensure that 
internal audit activity adheres to the IA Standards and rigorous 
methodology in delivering internal audits; and 


* conduct horizontal and sectoral audits in large and small 
departments and agencies. 


3.84 These responsibilities are designed to enhance the 
professionalization of the internal audit community. As part of our 
audit, we reviewed the actions of the OCG Internal Audit Sector. 
Our progress rating on the following topics can be found at the end 


of this section (see page 27). 


The Office of the Comptroller General provides leadership and guidance to the 
internal audit community 


3.85 Strategic direction and guidance. We conducted a number of 
interviews to identify the key mechanisms the OCG used to provide 
strategic direction and guidance to the internal audit community. We 
also reviewed guidance the OCG developed and steps it took to 
provide leadership to the internal audit community. 


- 3.86 We found that the OCG has carried out a wide range of activities 
to inform and offer direction to the community on changes to the 
Policy framework and the internal audit profession. These activities 
include developing guidance and providing training and strategic 
direction to the community. 


3.87. The OCG has developed a number of tools and guidance for 
internal audit activities. These include 


¢ guidance on developing risk-based audit plans, 
¢ guidance on core management controls, 


* an inventory of audit programs, 
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¢ models for internal audit and audit committee charters, 
* guidance on a practice inspection program, and 


* guidance for departmental audit committees. 


3.88 We also noted that the OCG has given strategic direction to the 
community. For example, as reported in our chapter on Canada’s 
Economic Action Plan in our 2010 Fall Report, the OCG provided 
effective leadership, direction, and guidance to departmental internal 
audit activities with respect to audit coverage of expenditures made 
under the terms of the Economic Action Plan. 


3.89 Professionalization of the internal audit activity. One of the 
OCG’s priorities has been to develop a human resource strategy to 
enhance professionalism in internal audit. This strategy focuses on 
continuing professional development and on creating a model that 
standardizes expectations, skills, and experience needed for internal 
auditors at various levels within departments. The strategy included 
working with departments to decide on how many internal auditors 
departments need. These approaches are aimed at defining the current 
state of capacity in the federal government and providing a basis for 
future development of the community. 


3.90 In conducting our audit, we interviewed a broad range of audit 
committee members, senior department managers, and the 
management and staff of internal audit groups. They raised several 
common concerns about what human resource strategies need to do 
for the internal audit activity. They agreed that such strategies need to 


* focus on ongoing professional development, 


* develop common expectations and classifications for the internal 
audit community to reduce the movement of internal audit staff 
from one organization to another and to achieve higher 
classification levels, and 


* provide for orderly career advancement within the federal 
government's internal audit community. 


3.91 We found that, in implementing the Policy, the OCG gave funds 
to departmental internal audit activities for the professional 
development of internal audit staff. As noted in Exhibit 3.8, ° 
departments have not used all of the funds they received for this 


purpose. 
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Exhibit 3.8 Departments did not use all of the funds they received for continuing professional 
development of internal audit staff 


2006-07 $2,455,000 $833,000 


2007-08 $3,334,000 $1,952,000 
2008-09 $3,026,000 $2,258,000 


3.92 We found that ongoing professional development is available. 
The OCG sponsors regular conferences for chief audit executives to 
address current issues and provide a forum for the exchange of 
information. The OCG also invites all members of the community to 
regular internal audit forums. We noted that the OCG has created 
working protocols with the Canada School of Public Service and the 
Ottawa chapter of the Institute of Internal Auditors to deliver training 
programs, such as a three-day internal audit orientation workshop. 


3.93 As part of our audit, we interviewed chief audit executives and 
internal audit staff about professional development. Many internal 
auditors said that the demands to complete assigned audits made it 
difficult to engage in ongoing professional development activities. 


3.94 In response to the issues related to classification and career or 
succession planning for the internal audit community, the OCG has 
designed a maturity model to be used to identify current capacity 
within the internal audit community and to serve as a roadmap for 
further development. The OCG will work with departments to 
identify the required number of internal audit resources by 
department. The OCG is in the process of implementing this model, 
which includes implementation of standard classifications across all 
government departments. 


3.95 The model includes key indicators for human resource 
development and levels of maturity that staff at various levels would be 
expected to show. The OCG has also created generic job descriptions 
for internal auditors at different levels within departments. The next 
step in developing and applying this model is to pilot test the model in 
12 departments in the 2011-12 fiscal year. 
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3.96 Monitoring. The OCG is required to monitor the 
implementation of the Treasury Board Policy on Internal Audit (the 
Policy) and find out whether its objectives are being met. The Policy 
requires the Treasury Board of Canada Secretariat to complete an 
evaluation of the Policy by 1 April 2011. We found that the OCG has 


begun this mandatory evaluation. 


3.97 As part of our audit, we reviewed the mechanisms the OCG has 
put in place to monitor how well the Policy’s requirements are being 
met and how effectively internal audit capacity is being developed 
across government. 


3.98 We noted that the main tool the OCG is using is the 
Management Accountability Framework. The OCG reviews and 
assesses information that departmental internal audit activities provide 
to find out how well they are complying with key aspects of the Policy. 
This process is designed to give the OCG an overview of the 
departments’ progress in implementing the Policy. 


3.99 Horizontal audits. In the 2009-10 fiscal year the OCG 
completed two horizontal audits in large departments and two in small 
departments and agencies. In previous years, such audits were not 
carried out beyond the survey phase. To support these audits, the OCG 
has created a government-wide, risk-based internal audit plan that is 
designed to identify the audits to be carried out over the next three 
years in both large departments and small departments and agencies. 
The OCG recently updated its risk-based plan and, in April 2010, 
presented the plan to its audit committee. In May 2010, departments 
received the approved horizontal audit plan, which sets out horizontal 
audits for the next three years. The risk-based plan also identifies areas 
of significance and risk in small departments and agencies that are to 
receive audit coverage. In 2010, the OCG Internal Audit Sector 
carried out two audits that addressed areas of risk in small agencies. 
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Exhibit 3.9 Progress in addressing our recommendations on staffing internal audit and audit coverage 


should establish benchmarks to determine the number of internal 


auditors that the federal government and each department need to 
provide a reasonable level of audit coverage and a sustainable 
audit function. Satisfactory 


(Recommendation 1.57 of the 2004 November Report of 
the Auditor General of Canada, Chapter 1, Internal Audit in 
Departments and Agencies) 


The Treasury Board Secretariat, in collaboration with departments, 
should determine the appropriate mix of qualifications, experience, 
and skills required for internal audit on a department and 

government-wide basis. Satisfactory 


(Recommendation 1.63 of the 2004 November Report of 
the Auditor General of Canada, Chapter 1) 


The Treasury Board Secretariat, in consultation with small entities, 
should develop a risk-based strategy and establish, within 
government, a capacity for providing internal audit services to 
small entities. Satisfactory 


(Recommendation 1.73 of the 2004 November Report of 
the Auditor General of Canada, Chapter 1) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Conclusion 


3.100 The government has shown satisfactory progress in acting upon 
the commitments it made in response to the observations and 
recommendations of our 2004 report. 


3.101 We found that independent departmental audit committees have 
been created in the 24 large departments, with the majority of 
members coming from outside the federal public administration. Audit 
committees are at various stages in developing their practices and 
procedures. Established departmental audit committees are having a 
positive impact by contributing to stronger internal audit in 
government. 


3.102 The government has strengthened its internal audit capacity 
within the large departments since our last audit, which was done 

in 2004. Departments have created internal audit charters that set out 
clear roles and responsibilities for internal audit activities. The 
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charters, along with strong support of senior management and 
departmental audit committees, have resulted in independent, 
objective internal audit activities. Most of the completed internal audit 
reports we reviewed are assuring management about internal control 
and making recommendations to improve departmental practices. 
Quality assurance and improvement processes for internal audits are at 
varying stages of development: some are well developed, while others 
are in early development. 


3.103 The Office of the Comptroller General of Canada has responded 
to the observations and recommendations of our 2004 report. The 
OCG introduced a new policy framework that requires it to create 
audit committees and set clear expectations for them. The OCG 
provides direction and guidance to the internal audit community and 
to departmental audit committees. 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these standards 
as the minimum requirement for our audits, we also draw upon the standards and practices of other disciplines. 


Objectives 


The overall audit objective was to assess the progress that the government has made in addressing the 
concerns we raised in Chapter | of our 2004 November Report. 


This follow-up audit assessed the extent to which, as required by the Treasury Board Policy on Internal 
Audit (the Policy), 


- deputy heads of departments and agencies have created independent departmental audit committees 
and these committees were carrying out the duties and responsibilities that the Policy and related 
guidance set out for them, 


* deputy heads of departments and agencies have appointed chief audit executives who are 
appropriately qualified and who have established internal audit activities, and 


* the Office of the Comptroller General of Canada (the OCG) is carrying out its duties and 
responsibilities to provide leadership and direction to the internal audit community. 


Scope and approach 


Departmental audit committees. The audit assessed whether deputy heads of departments and agencies 
had created an independent departmental audit committee and whether these committees had carried out 
the duties and responsibilities that the Policy and its related directives set out for them. 


We reviewed documentation for appointing external audit committee members and assessed whether, 
collectively, their backgrounds were consistent with the Policy’s requirements, particularly the 
requirement for financial expertise. 


Our audit included a review of the information that departments and agencies gave to audit committee 
members, to find out whether they received the information they needed to perform their roles and 
responsibilities. Our audit approach included interviews with a selection of audit committee members. 


The audit team assessed the nature and extent of the tools and training that audit committee members 
received to help them understand and carry out their obligations. 


This line of enquiry includes the following 24 large departments and agencies that represent approximately 
95 percent of the Government of Canada’s total assets, liabilities, revenues, and expenses: 


- Agriculture and Agri-Food Canada 
« Canada Border Services Agency 
« Canada Revenue Agency 


* Canadian Heritage 
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Citizenship and Immigration Canada 


° 


Canadian International Development Agency 
* Correctional Service Canada 
« Environment Canada 
- Finance Canada, Department of 
- Fisheries and Oceans Canada 
* Foreign Affairs and International Trade Canada 
* Health Canada 
« Human Resources and Skills Development Canada 
« Indian and Northern Affairs Canada 
¢ Industry Canada 
- Infrastructure Canada 
* Justice Canada, Department of 
« National Defence 
« Natural Resources Canada 
* Public Works and Government Services Canada 
* Royal Canadian Mounted Police 
- Transport Canada 
¢ Treasury Board and Office of the Comptroller General 
- Veterans Affairs Canada 
Departmental internal audit activities. We conducted quality assessment reviews of internal audit 
activities in a selection of six departments and agencies to find out if the activity met the requirements of 


the Institute of Internal Auditors International Professional Practices Framework (including the IIA 
Standards). The following six departments were judgmentally selected using a variety of criteria: 


* Canadian International Development Agency 

- Foreign Affairs and International Trade Canada 

* Health Canada 

* National Defence 

* Royal Canadian Mounted Police 

- Veterans Affairs Canada 
We also looked at the following four key aspects within the internal audit activity across a random sample 
of 12 of the 24 large departments and agencies: 

* reporting relationships 

- risk-based plans 

* reporting practices 


* completion of an independent quality assessment review of the activity 
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The 12 entities randomly selected for this review were the following: 


* Canada Revenue Agency 

* Canadian Heritage 

- Canadian International Development Agency 
¢ Correctional Service Canada 

- Environment Canada 


+ Finance Canada, Department of 


4 


¢ Foreign Affairs and International Trade Canada 


- Human Resources and Skills Development Canada 


* Justice Canada, Department of 


¢ Royal Canadian Mounted Police 


- Treasury Board of Canada Secretariat and the Office of the Comptroller General 


- Veterans Affairs Canada 


The Office of the Comptroller General of Canada. We assessed whether the OCG was providing 
leadership and direction to the internal audit and departmental audit communities. 


The audit excluded all Policy requirements related to small departments and agencies, except for the 


OCG’s requirement to conduct horizontal audits on these departments and agencies each year. 


Criteria 


Deputy heads are responsible for establishing an independent 
audit committee. 


¢ Policy on Internal Audit, section 5.4.2., Trea 


sury Board, 2009 


¢ Directive on Departmental Audit Committees, Treasury Board, 
2009 


Departmental audit committees are exercising oversight on 


departmental values and ethics, risk management, and 
management control frameworks, which aie reviewed with an 
appropriate risk-guided focus and cycle; 


the internal audit activity; 


results of work completed by the Office of the Auditor General 
and central agencies; 


follow-up on management action plans to address internal and 
external audit recommendations; 


financial statements and public accounts reporting; and 


risk and accountability reporting. 


Directive on Departmental Audit Committees, sections 4.1 and 
4.2, Treasury Board, 2009 


Departmental audit committees are meeting operational 
requirements of the Policy (charter, annual work plan, and 
annual report on activities). 


Directive on Departmental Audit Committees, section 4.4, 
Treasury Board, 2009 
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Deputy heads 


Guidelines on Expected Qualifications for Chief Audit 
Executives, section 3, Treasury Board 


have appointed chief audit executives at a senior executive 
level, reporting to the deputy head, with appropriate | ¢ Directive on Chief Audit Executives, Internal Audit Plans, and 
professional qualifications; and Support to the Comptroller General, section 4.1, Treasury 
Board, 2009 


Policy on Internal Audit, section 5.4.3 and 5.6.3, Treasury 
Board, 2009 


Institute of Internal Auditors International Professional 
Practices Framework, Institute of Internal Auditors, 2009. 


have ensured that the audit committees received the 
information they needed to carry out their responsibilities. 


Chief audit executives have met their responsibilities to 


——— + 


Institute of Internal Auditors International Professional 


; Practices Framework, Institute of Internal Auditors, 2009. 
establish appropriate policies and procedures to guide the — 


internal audit activity; 


Guidelines on Expected Qualifications for Chief Audit 


é : Executives, section 3, Treasury Board 
establish a risk-based audit plan that is done at least once a - 


year and that ensures proper coverage and minimizes 
duplication of effort; 


ensure that the Institute of Internal Auditors (IIA) International | 
Professional Practices Framework (including the IIA Standards) | 
are followed; 


ensure that internal auditors have appropriate professional 
qualifications, skills, and opportunities for training and 
development and obtain Certified Internal Auditor certification; | 


ensure that internal audit engagements are completed in a 
timely manner and that reports are given to the audit 


committee as quickly as possible; 


develop and maintain a quality assurance and improvement 
program that covers all aspects of the internal audit activity, 
and continuously monitor the program's effectiveness; 


ensure that a practice inspection or other external review of the 
internal audit activity is conducted at least every four years. 


The Comptroller General has met his or her responsibilities to * Policy on Internal Audit, Appendix—Responsibilities of the 
provide effective functional leadership by Comptroller General for Internal Audit, Treasury Board, 2009 
* developing and supporting the implementation of effective ¢ Policy on Internal Audit, sections 5.7, 5.8.4, 5.8.5, and 
internal auditing methodologies and procedures across 5.8.6, Treasury Board, 2009 
government; 


strengthening the internal audit community through 
recruitment and training programs; 


supporting the creation of independent departmental audit 
committees; 
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liaising with chief audit executives, deputy heads and agents 
of Parliament, the Public Service Commission, and other 
central agencies on significant audit issues; 


ensuring ongoing practice inspections and assessments of the 
entire spectrum of departmental internal audit activities; 


undertaking or leading horizontal audits across large and small 
departments and agencies; 


reporting periodically to Treasury Board on the overall state of 
risk management, control, and governance across government, 
with a fulsome report occurring at least every three years; 


supporting the implementation of appropriately qualified audit 
committees, as well as providing guidance on expected 
committee practices across government; and 


evaluating the Policy on Internal Audit by 1 April 2011. 


Management reviewed and accepted the suitability of the criteria used in the audit. 


Period covered by the audit 


The period covered by the audit was the fiscal year ending 31 March 2010. Audit work for this chapter was 
substantially completed on 28 November 2010. 


Audit team 


Assistant Auditor General: Nancy Y. Cheng 
Senior Principal: Bruce C. Sloan 

Principal: Karen Hogan 

Director: Marianne Avarello 


Daniel Boutin 
Caron Mervitz 
Sophie Miller 
Jacqueline Warren 


Judith Wilson 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 


Status Report of the Auditor General of Canada—2011 Chapter3 | 33 


INTERNAL AubIT 


Appendix List of recommendations 


The following is the recommendation found in Chapter 3. The number in front of the recommendation 


indicates the paragraph where it appears in the chapter. The numbers in parentheses indicate the 


paragraphs where the topic is discussed. 


Internal Audit 


3.80 Departments that have not had 
an external quality assessment should 
have an assessment conducted as 
required by the International Standards 
for the Professional Practice of Internal 
Auditing and the Treasury Board Policy 
on Internal Audit. (3.70-3.79) 
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The Treasury Board of Canada Secretariat’s response. Agreed. 
The Secretariat’s Internal Audit and Evaluation Bureau ([AEB) 
has planned for the conduct of an external quality assessment for 
the Secretariat as a department. Specifically, a self-assessment 
exercise was conducted in the 2010-11 fiscal year, in preparation 
for an external quality assessment to be completed by fall 2011. 


The Canada Border Services Agency’s response. Agreed. The 
Canada Border Services Agency intends to complete an external 
quality assessment by June 2012, as required by the International 
Standards for the Professional Practice of Internal Auditing and 
the Treasury Board Policy on Internal Audit. 


Canadian Heritage’s response. Agreed. Canadian Heritage notes 
the Office of the Auditor General’s recognition of quality 
assurance work conducted at the Department and will endeavour 
to undertake an external quality assessment every five years. 


Correctional Service Canada’s response. Agreed. We 
understand the necessity and usefulness of quality assessments. 
As such, we had a preliminary external assessment conducted in 
2006, where recommendations for improvement were made and 
actions have been taken to address them. The results of the 
assessment and related actions have been presented to and 
approved by the audit committee and reported annually in the 
Chief Audit Executive reports. At this point in time, following 
Treasury Board’s direction, Correctional Service Canada fully 
expects to have an external inspection conducted by the end 


of the 2011-12 fiscal year. 


The Department of Finance Canada’s response. Agreed. 
Internal audit and evaluation at the Department of Finance is 
currently planning to have an external quality assessment 
conducted, as required by the International Standards for 
Professional Practice of Internal Auditing and the Treasury 
Board Policy on Internal Audit, by September 2012. 
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Human Resources and Skills Development Canada’s 
response. Agreed. Corrective measures are well under way to 
address this observation. Prior to initiating a formal external 
quality assessment review in November 2010, the Internal Audit 
Services Branch conducted yearly self-assessments against the 
Institute of Internal Auditors Standards, the Office of the 
Comptroller General’s Internal Audit Self-Diagnostic Tool, and 
relevant Treasury Board policies. The Branch also engaged an 
independent external reviewer in the 2008—09 fiscal year to 
conduct a preliminary assessment of the Department’s readiness 
for the formal external assessment in the 2010-11 fiscal year. 
Building on the results of the preliminary assessment, the 
Department developed an action plan to address the gaps 
identified. The external quality assessment review is under way 


and will be completed by 30 April 2011. 


Industry Canada’s response. Agreed. An external quality 
assessment is currently planned for the 2013-14 fiscal year. 


The Department of Justice Canada’s response. Agreed. The 
Department of Justice is planning to conduct an external quality 
assessment during the 2011-12 fiscal year. 
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Broad concerns: Services 
to First Nations 


Programs for First Nations 
on Reserves 


Preface 


This preface contains views based on the numerous audits we have 
completed over the past decade. While not directly related to the audit 
that follows, the preface presents important matters that we believe 
should be brought to the attention of the House of Commons. 


Over the past 10 years, the Office of the Auditor General has audited 
a broad range of services and federal activities affecting First Nations. 
Throughout this period we have had the opportunity to visit many of 
Canada’s First Nations reserves, and to meet with chiefs, councils, and 
community leaders. We have interviewed hundreds of federal officials 
and numerous experts on First Nations issues. 


It is clear that living conditions are poorer on First Nations reserves 
than elsewhere in Canada. Analysis by Indian and Northern Affairs 
Canada (INAC) supports this view. The Department has developed 

a Community Well-Being Index based on a United Nations measure 
used to determine the relative living conditions of developing and 
developed countries. INAC uses its index to assess the relative progress 
in living conditions on reserves. In 2010, INAC reported that the 
index showed little or no progress in the well-being of First Nations 
communities between 2001 and 2006. Instead, the average well-being 
of those communities continued to rank significantly below that of 
other Canadian communities. Conditions on too many reserves are 
poor and have not improved significantly. 


In our audits, we have made numerous recommendations concerning 
federal programs and services for First Nations reserves. In our 

2006 May Report, Chapter 5, Management of Programs for First 
Nations, we found that progress was generally unsatisfactory in 
implementing the recommendations that are most important to the 
lives and well-being of First Nations people. In that audit, we also 
identified seven critical factors that appeared to have helped or 
hindered implementation of our recommendations, depending on their 
presence or absence (see paragraphs 5.50 to 5.61 of our 2006 Report 
and 4.7 of this report). 
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In the audit that follows, we have again found progress to be 
unsatisfactory on several recommendations we have made over the 
past decade that are important for the lives and well-being of First 
Nations people. Even where federal actions have led to the 
implementation of our recommendations, it appears to us that the 
results have not led to significant improvements in the lives and well- 
being of people living in many First Nations communities. 


Structural impediments explain the lack of progress on reserves 


In our view, many of the problems facing First Nations go deeper than 
the existing programs’ lack of efficiency and effectiveness. We believe 
that structural impediments severely limit the delivery of public 
services to First Nations communities and hinder improvements in 
living conditions on reserves. We have identified four such 
impediments: 


* lack of clarity about service levels, 

« lack of a legislative base, 

+ lack of an appropriate funding mechanism, and 

« lack of organizations to support local service delivery. 


Lack of clarity about service levels. Most of the services provided 

to communities throughout Canada are the responsibility of provincial 
and municipal governments, but this is not the case on reserves. 
Under the Constitution Act, 1867, the federal government has 
exclusive authority to legislate on matters pertaining to “Indians, 

and Lands reserved for Indians.” INAC has been the main federal 
organization exercising this authority. While the federal government 
has funded the delivery of many programs and services, it has not 
clearly defined the type and level of services it supports. 


Mainly through INAC, the federal government supports many services 
on reserves that are normally provided by provincial and municipal 
governments off reserves. It is not always evident whether the federal 
government is committed to providing services on reserves of the same 
range and quality as those provided to other communities across 
Canada. In some cases, the Department’s documents refer to services 
that are reasonably comparable to those of the provinces. But 
comparability is often poorly defined and may not include, for 
instance, the level and range of services to be provided. 


To provide true comparability, it would be important to include a clear 
statement of comparability in program objectives and define 
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comparability on a program-by-program basis. Roles and 
responsibilities would also need to be specified, as would the level of 
services required for comparability. In addition, the costs of achieving 
comparability would have to be determined and programs would have 
to be adequately funded. It would be necessary to establish measures 
for evaluating performance and determining whether the program was 
achieving the desired outcomes. 


Lack of a legislative base. Provincial legislation provides a basis of 
clarity for services delivered by provinces. A legislative base for 
programs specifies respective roles and responsibilities, eligibility, and 
other program elements. It constitutes an unambiguous commitment 
by government to deliver those services. The result is that 
accountability and funding are better defined. 


The federal government has often developed programs to support First 
Nations communities without establishing a legislative or regulatory 
framework for them. Therefore, for First Nations members living on 
reserves, there is no legislation supporting programs in important areas 
such as education, health, and drinking water. Instead, the federal 
government has developed programs and services for First Nations on 
the basis of policy. As a result, the services delivered under these 
programs are not always well defined and there is confusion about 
federal responsibility for funding them adequately. 


Lack of an appropriate funding mechanism. The federal government 
uses contribution agreements to fund the delivery of services on First 
Nations reserves. Through these agreements, First Nations receive a 
certain level of funding to provide various programs and services in their 
communities. We see several problems with the use of this funding 
mechanism for the provision of core government services. One problem 
is that, while the agreements state the services or actions to be provided, 
they do not always focus on service standards or results to be achieved. 


The timing for provision of funds under contribution agreements is 
also problematic. Most contribution agreements must be renewed 
yearly. In previous audits, we found that the funds may not be available 
until several months into the period to be funded; one reason is that 
new agreements cannot be finalized until departments have reviewed 
documentation and confirmed that funds from the previous period 
were used appropriately. Consequently, First Nations must often 
reallocate funds from elsewhere to continue meeting community 
service requirements. However, INAC recognizes the importance of 
providing certain services on an ongoing basis, such as health care and 
education. The Department therefore continues to fund these and 
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other essential services even before it has confirmed appropriate 
spending of funds for the previous period. 


The use of contribution agreements between the federal government 
and First Nations may also inhibit appropriate accountability to First 
Nations members. It is often unclear who is accountable to First 
Nations members for achieving improved outcomes or specific levels of 
services. First Nations often cite a lack of federal funding as the main 
reason for inadequate services. For its part, INAC maintains that the 
federal government funds services to First Nations but is not 
responsible for the delivery or provision of these services. 


Contribution agreements involve a significant reporting burden, 
especially for small First Nations with limited administrative capacity. 
Communities often have to use scarce administrative resources to 
respond to numerous reporting requirements stipulated in their 
agreements. We followed up on INAC’s efforts to reduce the reporting 
requirements of First Nations and found progress to date to be 
unsatisfactory even though the Department had taken various actions 
(see paragraphs 4.71 to 4.85). 


The use of contribution agreements to fund services for First Nations 
communities has also led to uncertainty about funding levels. 
Statutory programs such as land claim agreements must be fully 
funded, but this is not the case for services provided through 
contribution agreements. Accordingly, it is not certain whether 
funding levels provided to First Nations in one year will be available 
the following year. This situation creates a level of uncertainty for First 
Nations and makes long-term planning difficult. In contrast, legislation 
may commit the federal government to provide statutory funding to 
meet defined levels of service. A legislative base including statutory 
funding could remove the uncertainty that results when funding for 
services depends on the availability of resources. 


Lack of organizations to support local service delivery. Over the 
decades, provinces have established many organizations and structures 
to support local delivery of programs and services to communities. 

For example, provinces have developed school boards, health services 
boards, and social service organizations. These organizations can 
supply vital expertise, facilitate career advancement, and develop a 
means of efficient and efféctive delivery of services. 


There are few similar organizations to support service delivery within 
First Nations communities. The federal government established each 
First Nation band as an autonomous entity and provides separate 


4 Chapter 4 Status Report of the Auditor General of Canada—2011 


PROGRAMS FOR FIRST NATIONS ON RESERVES 


program funding to each. Many of these First Nations are small, 
consisting of communities that often have fewer than 500 residents. 
There are more than 600 First Nations across Canada. Many of them 
are hampered by the lack of expertise to meet the administrative 
requirements for delivering key programs within their reserves. They 
often do not have the benefit of school boards, health boards, or other 
regional bodies to support the First Nations as they provide services to 
community members. 


In an attempt to address the lack of organizations supporting the 
delivery of services, INAC has developed new approaches and worked 
with groups representing various First Nations. For example, the 
Department has worked with First Nations organizations and 
provincial governments to develop improved child and family service 
programs in several provinces. It has also recently launched the 
Reforming First Nation Education Initiative, which includes support 
for establishing partnerships with First Nations regional organizations 
and provinces; for example, it has provided funding to organizations 
such as the First Nations Education Steering Committee in British 
Columbia and the First Nations Education Council in Quebec. This is 
a start, but much remains to be done. 


Change is needed if meaningful progress is to be realized 


Despite the federal government’s many efforts to implement our 
recommendations and improve its First Nations programs, we have 
seen a lack of progress in improving the lives and well-being of people 
living on reserves. Services available on reserves are often not 
comparable to those provided off reserves by provinces and 
municipalities. Conditions on reserves have remained poor. Change is 
needed if First Nations are to experience more meaningful outcomes 
from the services they receive. We recognize that the issues are 
complex and that solutions will require concerted efforts of the federal 
government and First Nations, in collaboration with provincial 
governments and other parties. 


We believe that there have been structural impediments to 
improvements in living conditions on First Nations reserves. In our 
opinion, real improvement will depend on clarity about service levels, 
a legislative base for programs, commensurate statutory funding 
instead of reliance on policy and contribution agreements, and 
organizations that support service delivery by First Nations. All 

four are needed before conditions on reserves will approach those 
existing elsewhere across Canada. There needs to be stronger emphasis 
on achieving results. 
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We recognize that the federal government cannot put all of these 
structural changes in place by itself since they would fundamentally 
alter its relationship with First Nations. For this reason, First Nations 
themselves would have to play an important role in bringing about the 
changes. They would have to become actively engaged in developing 
service standards and determining how the standards will be 
monitored and enforced. They would have to fully participate in the 
development of legislative reforms. First Nations would also have to 
co-lead discussions on identifying credible funding mechanisms that 
are administratively workable and that ensure accountable governance 
within their communities. First Nations would have to play an active 
role in the development and administration of new organizations to 
support the local delivery of services to their communities. 


Addressing these structural impediments will be a challenge. The 
federal government and First Nations will have to work together and 
decide how they will deal with numerous obstacles that surely lie 
ahead. Unless they rise to this challenge, however, living conditions 
may continue to be poorer on First Nations reserves than elsewhere in 
Canada for generations to come. 
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The federal government supports numerous services to First Nations 
members on reserves that are similar to those provided by provincial 
and municipal governments to people off reserves. These services 
range from education and social development programs to land 
management and economic development, housing, provision of 
potable water, and provision of benefits to First Nations members 
under treaties and other agreements. 


In several audits over the past decade, we have identified issues of 
particular importance to the lives and well-being of First Nations and 
Inuit. In this follow-up audit, we examined the government’s progress 
toward achieving the commitments it made to address significant 
observations and recommendations from seven of those reports, issued 
between 2002 and 2008. We focused on the areas of education, water 
quality, housing, child and family services, land claim agreements, and 
reporting requirements. 


Our audit included the roles that Indian and Northern Affairs Canada 
(INAC), Health Canada, the Canada Mortgage and Housing 
Corporation (CMHC), and the Treasury Board of Canada Secretariat 
play in the management of programs to improve the lives of First 
Nations and Inuit. 


Audit work for this chapter was substantially completed 
on | November 2010. 


According to the Report of the Royal Commission on Aboriginal 
Peoples (1996), Canada’s First Nations reserves experience long- 
standing challenges such as ill health, insufficient and unsafe housing, 
high unemployment, polluted water supplies, inadequate education, 
poverty, and family breakdown. The federal government supports 
programs that can improve the lives of people who live on reserves. 
However, despite the billions of federal dollars spent each year on 
programs to address the social and economic challenges on First 
Nations reserves, conditions there remain significantly below the 
national average. 
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What we found = Despite considerable efforts and concrete actions in some areas, 
federal organizations have not made satisfactory progress on 
commitments made in response to several recommendations from 
our previous audits. INAC has just begun implementing a strategy to 
close the education gap. The Department has not defined what is 
meant by its policy of providing child and family services that are 
reasonably comparable to what exists in the provinces. Nor has it 
developed and implemented a plan to communicate to other federal 
organizations what specific obligations they have under land claim 
agreements or monitored their results. INAC and Health Canada do 
not ensure that drinking water is tested on a regular basis. INAC, 
Health Canada, and the Canada Mortgage and Housing Corporation 
have developed a strategy for dealing with mould in housing, but it is 
not comprehensive as it focuses on education rather than 
remediation and provides no new funding. 


* INAC has implemented commitments it made in response to some of 
our recommendations. For example, it has taken steps to monitor 
progress toward the objectives of comprehensive land claim 
agreements, and has put in place a system to track and monitor 
progress on all federal commitments contained in the agreements. 
Along with Health Canada, INAC has developed draft legislation 
related to drinking water on reserves. In addition, the Department 
recently agreed with several provinces and First Nations on 
frameworks to improve child and family services on reserves, but it 
will likely be several years before meaningful results are realized. 


Notwithstanding the considerable efforts made, conditions have 
generally not improved for First Nations in each of the areas 

subject to our audit. The education gap between First Nations 
living on reserves and the general Canadian population has widened, 
the shortage of adequate housing on reserves has increased, 
comparability of child and family services is not ensured, and the 
reporting requirements on First Nations remain burdensome. 


Broader concerns that we believe have inhibited progress include 
the lack of clarity about service levels on First Nations reserves, lack 
of a legislative base to fund service delivery on reserves, a lack of an 
appropriate funding mechanism, and a lack of organizations that 
could support local service delivery. There is a risk that living 
conditions on many First Nations reserves will remain significantly 
below national averages, with little prospect of a brighter future, 
until these concerns are addressed. 


The Department has responded. The Department agrees with our 
recommendation. Its detailed response follows the recommendation. 
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Introduction 


41 The federal government funds and supports many programs and 
services to First Nations communities that are similar to the services 
provided by provinces and municipalities off reserves. Under section 91 
of the Constitution Act, 1867, Parliament has legislative jurisdiction over 
“Indians, and Lands reserved for Indians.” The Indian Act, passed under 
this authority, and numerous court decisions in this area establish a 
complex legal regime that applies to reserves and Status Indians. INAC 
has been the main federal organization exercising this authority. While 
the federal government has funded and supported many programs and 
services, it has not clearly defined the type and level of services 

it supports. 


4.2 The programs and services include education and social 


. development programs, land management and economic development, 


housing, drinking water, and the provision of benefits to First Nations 
members under treaties and other agreements. Approximately half of 
the Status Indians in Canada live off reserves and receive some health 
and education benefits provided by the federal government. 


4.3. First Nations members generally face far greater challenges than 
those confronting Canadian society as a whole, whether they live on 
or off reserves. The challenges include lower-than-average education 
levels, higher unemployment, and poorer quality housing and 
infrastructure. Tuberculosis infection rates are much higher among 
First Nations members than among other Canadians. 


44 The Royal Commission on Aboriginal Peoples referred to many 
of these challenges faced by First Nations when it reported in 1996. 

It presented numerous recommendations to improve the relationship 
between the federal government and First Nations. The government 
responded in 1997 with its report, Gathering Strength: Canada’s Aboriginal 
Action Plan. The Plan identified four key areas of focus: renewing 
partnerships, strengthening Aboriginal governance, developing a new 
fiscal relationship, and supporting strong communities. In 2000, the 
government's progress report on the Plan claimed impressive results. 
In contrast, the Assembly of First Nations published a report card 

in 2006 stating that Canada had failed in its actions responding to 
the recommendations of the Royal Commission. 


45 The Auditor General of Canada established Aboriginal issues 
as an important focus area for performance audits. In its reports to 
Parliament between 2001 and spring 2010, the Office of the Auditor 
General published 16 chapters addressing First Nations and Inuit 
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issues directly. Another 15 chapters dealt with issues of importance 
to Aboriginal people. The Office made numerous recommendations 
calling on Indian and Northern Affairs Canada (INAC) and other 
federal departments to address a wide range of issues of importance 
to First Nations and Inuit people. 


What we found in 2006 


46 In our 2006 May Report, Chapter 5, Management of Programs for 
First Nations, we followed up on progress made toward addressing the 
recommendations we had issued in seven previous reports. We found 
substantial progress in some areas and less in others. We particularly 
found unsatisfactory progress on 15 recommendations—generally those 
most likely to improve the lives of First Nations living on reserves. 
Departments had made satisfactory progress in addressing 22 other 
recommendations; many of these tended to be more administrative in 
nature and to have less direct impact on the lives of First Nations people. 


47 Critical factors. Our 2006 May Report also identified 
seven factors that we regarded as critical to departments’ success in 
implementing our recommendations: 


* sustained management attention, 

* coordination of government programs, 

* meaningful consultation with First Nations, 

* developing capacity within First Nations, 

* establishing First Nations institutions, 

* an appropriate legislative base for programs, and 


* managing the conflicting roles of Indian and Northern 
Affairs Canada. 


48 We concluded that full consideration of these factors when 
adjusting existing programs and implementing new ones would make 
a significant difference in the lives of First Nations people. The 
Government of Canada agreed that the critical factors were important 
and stated that they constituted an increasingly important part of 

the government's approach for the broader Aboriginal agenda. The 
government committed to taking the critical factors into account 
when developing approaches aimed at securing a better future for 
Aboriginal peoples. 
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Focus of the audit 


49 This follow-up audit examines progress made by federal 
departments in addressing selected observations and recommendations 
from seven previous reports completed between 2002 and 2008. The 
reports covered a wide range of issues, all of which we consider to be 
important to the lives and well-being of First Nations people. The 
issues include the following: 


* Education. In 2000 and 2004, our reports identified a gap 
between the secondary school completion rates for First Nations 
people on reserves and the rates for other Canadians. 


¢ Water. In 2005, we reported the lack of a legislative regime to 
ensure that water quality on reserves met the Guidelines for 
Canadian Drinking Water Quality, despite the existence of such 
a regime in every province and territory. 


* Housing. In 2003, we reported a significant housing shortage 
on reserves and the need for major renovations of about 
44 percent of existing housing because of problems such as mould 
contamination. In 2006, we reported unsatisfactory progress in 
addressing the problem of mould. 


* Child and family services. In 2008, we reported that First 
Nations children were eight times more likely to be removed 
from their homes than other Canadian children. 


* Land claim agreements. In 2003 and 2007, we reported that 
the federal government was not implementing all of its obligations 
under land claim agreements and was not living up to the spirit 
and intent of the agreements. 


¢ Reporting requirements. In 2002, we noted that First Nations 
communities, many of them having fewer than 500 members, 
had to fill out an excessive number of reports for INAC each year, 
and that many of the reports were never reviewed and served 
no purpose. 


4.10 This audit follows up on key issues in each of these areas and 
examines the extent to which federal organizations have addressed 
related recommendations and have acted on their commitments. It 
examines actions taken by INAC, Health Canada, the Canada 
Mortgage and Housing Corporation, and the Treasury Board of 
Canada Secretariat to address 12 of the 50 recommendations that 
directly relate to the key issues we examined in chapters of 

seven previous reports to Parliament. Where applicable, it also 
provides an update on actions taken in relation to the government’s 
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Progress on implementation 
of recommendations 


Chapter 4 


commitments in response to recommendations made by the House 
of Commons Standing Committee on Public Accounts. 


4.11 The preface identifies underlying concerns that we believe 
have inhibited more meaningful progress. 


4.12 More details about the audit objectives, scope, approach, 
and criteria are in About the Audit at the end of this chapter. 


Observations and Recommendation 


4.13 Even after considerable efforts and action on the part of the 
federal government, progress has been elusive on many of the issues 
raised in previous reports of the Office of the Auditor General 
concerning First Nations people living on reserves. We found 
unsatisfactory progress in addressing several of the recommendations 
on which we followed up, including in the areas of education, housing, 
child and family services, and the reporting requirements placed on First 
Nations. On the positive side, we found satisfactory progress in the steps 
taken thus far to address recommendations concerning the introduction 
of a regulatory regime for drinking water on reserves and improved 
implementation of comprehensive land claim agreements. Nevertheless, 
progress overall has been slow and has yet to significantly improve 
conditions in First Nations communities. A recommendation regarding 
our previous recommendations on which progress to date has been 
unsatisfactory or incomplete appears at paragraph 4.86. 


Efforts to close the education gap have yet to yield results 


4.14 Education is critical to raising the social and economic strength 
of First Nations individuals and communities to a level reached by 
other Canadians. Many First Nations students and communities face 
fundamental issues and challenges that are comparatively rare among 
other Canadians and that may impede their educational achievement. 
We found that, based on 2001 and 2006 census data, the education 
gap has not been reduced and Indian and Northern Affairs Canada 
(INAC) has only begun to implement a strategy for closing it. 


4.15 In 2004, we reported a significant gap in educational 
achievement, measured by secondary school graduation rates, between 
First Nations members living on reserves and the overall Canadian 
population. We noted that INAC used a funding formula dating back 
to the 1980s and lacked information that would enable it to compare 
costs with those of providing comparable education services in the 
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provinces. Consequently, the Department did not know whether the 
funding it provided to First Nations was appropriate. Concerning post- 
secondary education, we found that the funding allocation mechanism 
did not ensure equitable access to financial support for as many 
students as possible. 


4.16 Education gap. We followed up on two recommendations in 
our 2004 audit of INAC’s education program and post-secondary 
student support. We also examined actions taken on the government’s 
commitments in response to a recommendation of the House of 
Commons Standing Committee on Public Accounts (the Public 
Accounts Committee) related to post-secondary education. In 
particular, in our 2004 audit we recommended that INAC develop 
and implement a comprehensive strategy and action plan to close the 
education gap. INAC started work to respond to the recommendation, 
but we found that it has not maintained a consistent approach and 
cannot demonstrate improvements to date. It did not fully implement 
the action plan drafted in response to our audit. 


4.17 Meanwhile the proportion of high school graduates has risen 
steadily in the general population across Canada but not among First 
Nations students living on reserves. Based on census data from 2001 
and 2006, the education gap is widening. The proportion of high 
school graduates over the age of 15 is 41 percent among First Nations 
members living on reserves, compared with 77 percent for Canadians 
as a whole. In 2004, we noted that at existing rates, it would take 

28 years for First Nations communities to reach the national average. 
More recent trends suggest that the time needed may be still longer. 


4.18 In 2008, INAC launched the Reforming First Nation Education 
Initiative, consisting of two programs. The target is to have 75 percent of 
First Nations students achieve education outcomes comparable to the 
rest of Canada by 2028. The initiative is intended to, among other things, 
foster more partnerships with the provinces. It is too early to assess the 
initiative’s results since the new programs are just starting. More than 
six years after we last recommended that INAC implement a strategy to 
close the education gap, we found that it has only begun to do so. 


4.19 Cost comparisons. In 2004, we recommended that INAC 
determine and compare the costs of delivering education services on 
reserves with the costs of providing comparable education services in 
the provinces. INAC undertook to perform a review comparing the 
funds it allocates to schools operated by First Nations with the funds 
those schools would receive under the applicable provincial funding 
formula. We believe that knowing these costs is necessary for 
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developing an effective education strategy, closing the education gap, 
and meeting the education needs of students living on reserves. INAC 
spent $1.8 billion on its education program in the 2008-09 fiscal year. 


4.20 We found that INAC has reviewed its funding for First Nations 
schools. The Department compared the formula it used to fund First 
Nations schools with the formulas used by several provinces. It estimated 
that per-student funding was similar, although there were regional 
differences. The Department also identified seven categories of factors 
having a significant impact on the cost of First Nations education, 
although it did not make funding adjustments based on its findings. 


4.21 Post-secondary funding. In line with our 2004 audit, the Public 
Accounts Committee recommended that INAC review its post- 
secondary funding mechanisms, noting that existing mechanisms did 
not ensure equitable funding to as many eligible students as possible. 
The government responded that INAC was undertaking a 
comprehensive review of its post-secondary education program, 
including an examination of its funding mechanisms. Although the 
Department has studied various delivery options for post-secondary 
programs, we found that it has not specifically reviewed post-secondary 
funding mechanisms. As in 2004, INAC still allocates funds by First 
Nations community without regard to the number of eligible students; 
moreover, band governments have the flexibility to allocate the funds 
outside the program. Again, as in 2004, we found that the current 
funding mechanism and delivery model used to fund post-secondary 
education does not ensure that eligible students have equitable access 
to post-secondary education funding. 


4.22 More than six years after our previous audit, we found that 
INAC has taken various actions but has not maintained a consistent 
approach to education on reserves. It has yet to make progress in 
closing the education gap (Exhibit 4.1). This could have significant 
consequences for First Nations members living on reserves because 
they might not have access to the benefits associated with a higher 
level of education. Post-secondary education could improve 
employment opportunities for First Nations. 


Federal action on drinking water quality has not led to significant improvements 


4.23 Access to safe drinking water is vital to the health of all 
Canadians. Drinking water needs to be managed rigorously and tested 
regularly to ensure its safety and protect public health. Provinces have 
legislation and regulations in place to ensure the safety of the drinking 
water supply chain, but provincial regulations are not applied on 
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Exhibit 4.1 Progress on addressing two recommendations on education 


Indian and Northern Affairs Canada, in consultation with First 
Nations, should immediately develop and implement a 
comprehensive strategy and action plan, with targets, to close the 
education gap. It should also report progress to Parliament and to 
First Nations on a timely basis. Unsatisfactory 


(Recommendation 5.33 of the 2004 November Report of the 
Auditor General of Canada, Chapter 5. Indian and Northern Affairs 
Canada—Education Program and Post-Secondary Student Support) 


Indian and Northern Affairs Canada should undertake to obtain 
reliable and consistent information on the actual costs of delivering 
education services on reserves and compare the costs with those 
of providing comparable education services in the provinces. Satisfactory 


(Recommendation 5.51 of the 2004 November Report of the 
Auditor General of Canada, Chapter 5) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


reserves. INAC and Health Canada have developed the proposed 
legislation that was before Parliament at the end of the audit, but there 
are still no legislated standards for water quality on reserves. According 
to INAC data, more than half of the drinking water systems on 
reserves continue to pose a risk to the people who use them. 


4.24 In 2005, we audited drinking water on reserves and found that 
First Nations communities did not benefit from a level of drinking water 
protection comparable to that available to people living off reserves 
because provincial legislation and regulations are not applied on 
reserves. The 2005 audit found that the federal government had not 
acted even though it had primary jurisdiction to regulate water supply 
on reserves. Consequently, First Nations communities did not have a 
regulatory regime to govern drinking water. INAC and Health Canada 
instead used various funding arrangements with First Nations and 
monitored activities through administrative reporting requirements 
associated with the contribution funding. There were no statutes or 
regulations to require monitoring of the quality and safety of drinking 
water on reserves. Our 2005 audit also found that, in most First Nations 
communities, drinking water was tested less frequently than 
recommended under the Guidelines for Canadian Drinking Water 
Quality. We noted as well that INAC had found a significant risk to 
the quality or safety of drinking water in three quarters of the drinking 
water systems on reserves. 
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4.25 In this follow-up, we assessed the progress made in implementing 
two of our recommendations and in addressing the water systems 
posing significant risks. We examined actions taken by INAC and 
Health Canada to implement a regulatory regime. We also assessed 
whether INAC had reduced the number of drinking water systems 
posing a significant risk to the safety of drinking water. 


4.26 Regulatory regime. In 2005, we recommended that INAC and 
Health Canada implement a regulatory regime for drinking water in 
First Nations communities. INAC responded that, together with Health 
Canada and in consultation with First Nations, it would fully explore 
the options and feasibility of a regulatory regime. The government has 
since committed to developing such a regime. We found that INAC and 
Health Canada have made some progress in this direction. Following a 
consultation process started in 2006 with First Nations, INAC and 
Health Canada drafted a bill tabled in Parliament in May 2010. The bill 
would enable the government to make regulations related to drinking 
water on reserves, and to incorporate by reference corresponding 
provincial legislation and adapt it to meet the needs of First Nations. 
At the end of our audit, the bill was at second reading. 


4.27 Even after the bill is passed into law, however, it may still take 
years to develop and implement related regulations. INAC and Health 
Canada plan to phase them in following consultation, negotiation, and 
capacity development. After this, the two departments plan to seek 
funding for developing and implementing regulations and compliance 
plans. INAC plans to absorb the cost of enforcing regulations from its 
existing resource allocations. It may, therefore, be years before a fully 
implemented regulatory regime on drinking water is in effect for all 
First Nations reserves. 


4.28 Water quality monitoring. The 2005 audit also recommended 
that Health Canada, in consultation with INAC, ensure that all 
drinking water tests be carried out, that test results be properly 
recorded and shared, that situations where drinking water was not safe 
be identified and the necessary measures for each situation be defined, 
and that action be taken by responsible parties as required. The 
departments agreed with the recommendation and committed to 
improving their procedures to address these issues. We found that 
Health Canada and INAC have taken action to improve their 
administrative procedures to address each of the issues. Since the 2005 
audit, the departments have reviewed, clarified, and formalized their 
roles and responsibilities for managing drinking water systems: The 
departments also improved their guidance to communities and 
established electronic management systems in most regions. 
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4.29 We also examined specific actions taken to ensure that all 
drinking water tests were carried out and that the results were recorded 
and shared by Health Canada and INAC. For this purpose, we selected 
20 First Nations community water systems from three regions where 
advisories on drinking water quality were issued between January 2009 
and March 2010. We found that the departments did not fully 
implement many of the requirements we examined. For example, 
between 2006 and 2010 INAC had conducted only 25 of 80 required 
annual inspections and 47 of 80 risk evaluations in our sample. For 
the 2008-09 fiscal year, Health Canada reports that its data indicated 
that water at First Nations community sites was tested less often than 
recommended under departmental procedures, which are based on the 
Guidelines for Canadian Drinking Water Quality. At only 40 percent of 
community sites, for example, bacteriological sampling was performed 
at the recommended frequency. 


4.30 We found that in one of the three regions in our sample, Health 
Canada did not systematically share information with INAC on water 
test results as required. Health Canada told us that it could not do so 

because an information-sharing requirement had not been included in 
its funding arrangements with tribal councils. 


4.31 We also examined actions taken by Health Canada to ensure 
that situations where drinking water was not safe were identified, the 
necessary measures for each situation were defined, and action was 
taken as required. We found that Health Canada had informed First 
Nations when test results showed that drinking water was not safe and 
had recommended actions to be taken by the First Nations. We found 
as well that Health Canada and INAC established a process to ensure 
that appropriate actions were taken in these situations. Further, we 
found that Health Canada monitors most water systems for which 
drinking water advisories have been issued. 


4.32 Inthe absence of a regulatory enforcement regime, INAC 
included its procedures in the terms and conditions of contribution 
agreements with First Nations. Health Canada took a different 
approach, simply considering its procedures to be best practices. 
We noted that a Health Canada internal memorandum reported 
that since the 2004-05 fiscal year, the Pacific, Saskatchewan, and 
Ontario regions have not achieved 75 percent of the recommended 
bacteriological sampling frequency for piped community systems. 
We note that beginning in April 2010, Health Canada started to 
include its procedures in some of its funding agreements with 

First Nations. 
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4.33 Risks posed by drinking water systems. INAC uses results 

of annual inspections and risk evaluations to rate First Nations 
communities’ drinking water systems as high, medium, or low risk. 

As mentioned earlier, we found that INAC had not completed annual 
inspections and risk evaluations for most of our sample. In its 2006-07 
Report on Plans and Priorities, INAC had planned to reduce all high- 
risk systems and the majority of medium-risk systems by April 2008. 
INAC reported that it had reduced the proportion of high-risk systems 
from 29 percent to 6 percent by March 2010, while the proportion of 
medium-risk systems rose from 46 percent to 53 percent. The figures 
may not be accurate because the ratings were not always based on up-to- 
date inspections. INAC reported that more than half of drinking water 
systems on reserves still pose a significant risk to community members. 


4.34 First Nations reserves may still be years away from having 
drinking water protection comparable to what exists off-reserve in 
Canada. As of March 2010, more than half of water systems on reserves 
still posed a medium or high risk to the community members they 
served. There are still no legislative or regulatory regimes applicable to 
First Nations reserves, and putting these in place will take time and 
collaborative efforts. Until they are in place, there will be no statutory 
enforcement powers for water quality on reserves. Until then, INAC 
and Health Canada cannot be sure that First Nations members living 
on reserves have ongoing access to safe drinking water (Exhibit 4.2). 


Federal housing initiatives are not keeping pace with needs 


4.35. Although the federal government takes the view that it does not 
provide housing support on reserves as a result of legislative or treaty 
obligations, INAC and the Canada Mortgage and Housing 
Corporation (CMHC) assist First Nations in meeting their on-reserve 
housing needs by providing funding. First Nations have responsibilities 
related to housing. In 2003, we reported that, according to INAC, the 
housing stock generally deteriorates more rapidly on reserves, mainly 
because of substandard construction practices or materials, lack of 
proper maintenance, and overcrowding. We noted that poor housing 
on reserves has been shown to have a detrimental effect on the health, 
education, and overall social conditions of First Nations members and 
communities. Moreover, the population on reserves has been growing 
faster than the general population, heightening the need for more 
housing. For several years, mould contamination has been identified as 
a serious health and safety problem in First Nations reserves, liable to 
cause respiratory illnesses such as asthma. In this audit, we found that 
housing conditions on reserves are worsening. We also found that 
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Exhibit 4.2 Progress on addressing two recommendations on drinking water 


sy “= 

Indian and Northern Affairs Canada and Health Canada, in 
consultation with First Nations, should develop and implement a 
regulatory regime for drinking water in First Nations communities. 
This regime should be comparable with that in provinces and 
designed to protect the health and safety of First Nations people. 

_ As a minimum, this regime should deal with roles and 
responsibilities, water quality requirements, technical Satisfactory 
requirements, certification of systems and operators, compliance 
and enforcement, and public reporting requirements. 


(Recommendation 5.36 of the 2005 September Report of the 
Commissioner of the Environment and Sustainable Development, 
Chapter 5, Drinking Water in First Nations Communities) 


Health Canada, in consultation with Indian and Northern Affairs 
Canada and First Nations, should ensure that 


e 


all drinking water tests recommended under the Guidelines for 
Canadian Drinking Water Quality are being carried out; 


test results are properly recorded and relevant information is 
shared with appropriate parties; 


situations where drinking water is not safe, or where there is no Unsatisfactory 
assurance of drinking water safety, are clearly identified and 
actions to be taken in each situation are defined; and 


action is taken by responsible parties as required. 
(Recommendation 5.69 of the 2005 September Report of the 
Commissioner of the Environment and Sustainable Development, 
Chapter 5) 


| 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


federal organizations have not taken significant direct actions to 
remediate mould contamination, although they have made efforts to 
increase awareness and training. 


4.36 We audited housing in 2003. We noted then that INAC estimated 
a shortage of about 8,500 housing units on reserves. We also noted that 
about 44 percent of the existing housing required renovations. We raised 
many issues in the audit and followed up on them in 2006. We found 
progress in some areas, such as the implementation of a housing policy, 
but we also found an ongoing serious problem of mould in many houses 
on reserves. In 2003, INAC, Health Canada, and CMHC committed to 
taking action to address mould. Despite this, our 2006 audit found that 
all three federal organizations had not acted. They had neither assessed 
the extent of the mould problem nor taken responsibility for developing 
a comprehensive strategy to address it. 
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4.37 In this audit, we re-examined the extent of the housing shortage 
and the number of houses in need of significant renovations to protect 
the occupants’ health and safety. We also examined whether INAC, 
Health Canada, and CMHC had implemented our recommendation to 
develop and implement a mould strategy. 


4.38 Housing construction and renovations. We found that INAC 
and CMHC have made new investments in housing since our 2003 


and 2006 audits (Exhibit 4.3). 


Exhibit 4.3 INAC and CMHC have made significant investments in housing on reserves 


Funding ($ billions) 


Number of new units constructed 


Number of units receiving major renovations 


Note: The figures shown are cumulative from April 2004 to March 2009. We did not audit them. 
Source: INAC and CMHC data 


4.39 We note, however, that the investments have not kept pace with 
either the demand for new housing or the need for major renovations 
to existing units. According to unaudited data provided by INAC, the 
housing shortage on reserves has worsened since our 2003 audit 
because of increases in the demand for housing, the number of housing 
units having to be replaced, and the number of units requiring 
significant renovations for health and safety reasons. At the same time, 
since 2003, the average cost to build or renovate each house has risen 
significantly (Exhibit 4.4). In the 2008-09 fiscal year, new houses 
constructed on reserves amounted to only 30 percent of the existing 
houses that needed to be replaced. 


Exhibit 4.4 Housing requirements and costs have risen 


Demand for housing on reserves 


5,199 5,480 ; 


$42,750 $64,000 


Source: INAC data and calculations based on that data 


Housing units requiring replacement 


Housing units requiring major renovations 40% 


Average cost per house (constructed or 
significantly renovated) 


Note: We did not audit the figures shown. 
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4.40 INAC also administers a loan guarantee program for housing. 
Because the land on reserves belongs to the Crown, lending 
institutions cannot secure loans by conventional mortgages. The loan 
guarantee program enables individual First Nations members to obtain 
secured loans from lending institutions so that they can construct new 
homes or purchase existing ones. The program secured guarantees for 
an additional 400 individual home loans between April 2004 and _ 
March 2009. We also note that Budget 2007 set aside $300 million to 


support the development of individual home ownership on reserves. 


4.41 Mould. In 2003, we recommended that INAC, Health 

Canada, and CMHC, in consultation with First Nations, develop a 
comprehensive strategy and action plan to address the problem of mould 
on reserves. The three organizations agreed with the recommendation. 
In our 2006 follow-up audit, we found that the recommendation had not 
been implemented. 


4.42 In this audit, we found that progress was still not satisfactory. 
After consultations with First Nations, in 2008 the three organizations 
developed and approved a National Strategy to Address Mould in First 
Nations Communities. CMHC considers that the mould strategy is 
comprehensive because it involves the key parties, including First 
Nations and the three federal organizations; it also outlines respective 
roles, provides information on how to address mould, and promotes 
existing programs that can be used for mould prevention and 
remediation. 


4.43 Since adoption of the strategy, the organizations have worked to 
increase awareness of the causes of mould on reserves and provide 
more training for addressing the problem. For example, Health Canada 
developed awareness materials, such as a poster and booklet; it also 
tracks the number of visitors to its website. However, we found that 
the three federal organizations had not allocated additional funding to 
address mould as a result of the strategy, had not determined the 
magnitude of the problem, had not estimated the costs to remediate 
existing mould, had not completed most action items in the strategy, 
and had not met the strategy’s timelines. Moreover, because the 
organizations had not developed performance indicators, they did not 
know whether the strategy was making a difference. We also note that 
the strategy focuses on education to prevent mould but does not 
address rernediation of houses that are already contaminated. Further, 
the strategy does not address overcrowding, which contributes 
significantly to the problem of mould on reserves. 
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4.44 We note as well that the mould strategy has not been directly 
linked to renovation funding, although it provides for INAC and 
CMHC to help First Nations identify resources available from other 
sources to address mould. Nonetheless, some of the funding for 
renovating homes on reserves can be used to address mould while work 
is proceeding on other renovation projects. INAC and CMHC have 
funded the renovations of over 13,000 housing units on reserves since 
our 2003 audit (Exhibit 4.3). The federal government has committed 
an additional $400 million over two years under Canada’s Economic 
Action Plan to build and renovate housing on reserves. Many of the 
renovations may have addressed mould issues, but the Department 
does not track remediation actions to address mould. 


4.45 The adequacy of housing on reserves can affect the health, 
education, and socio-economic outcomes of First Nations members. 
In this audit, we observed that, according to INAC data, the number 
of housing units constructed and renovated was not keeping pace with 
the need, and conditions were deteriorating. We also found that 
INAC, Health Canada, and CMHC have developed a strategy to 
address the problem of mould on reserves, but it is not funded and does 
not identify or deal with the cost of remediation requirements. While 
the organizations have provided education and health promotion 
activities under the strategy, First Nations members living on reserves 
may continue to experience negative impacts on their health and 
social conditions. Eight years after our recommendation to develop 
and implement a mould strategy, we assess federal organizations’ 
progress as still unsatisfactory (Exhibit 4.5). 


Exhibit 4.5 Progress on addressing a recommendation on housing 


Indian and Northern Affairs Canada, Canada Mortgage and 
Housing Corporation, and Health Canada, in consultation with 
First Nations, should develop a comprehensive strategy and action 
plan to address the problem of mould on reserves. 


(Recommendation 6.48 of the 2003 April Report of the Auditor 
General of Canada, Chapter 6, Federal Government Support to 
First Nations—Housing on Reserves) 


Unsatisfactory 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 
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INAC has taken action on child and family services but has not defined its policy 
commitment to provide comparable services 


4.46 First Nations children are among the most vulnerable members 
of society. In 2008, we noted that over five percent of all children 
residing on reserves were in care; this was close to eight times the 
proportion of children residing off reserves. INAC has taken some 
actions to implement the two recommendations on which we followed 
up for this audit. Nevertheless, there has yet to be a notable change 
in the number of First Nations children in care. 


4.47 In 2008, we audited INAC’s program for child and family 
services on reserves. We found that INAC had not defined key policy 
requirements related to culturally appropriate child and family services 
and comparability of services with those provided by provinces. 
Moreover, the Department had no assurance that its First Nations 
Child and Family Services Program funded child welfare services that 
were culturally appropriate or reasonably comparable with those 
normally provided off reserves in similar circumstances. We also found 
that there was no link between the financial obligations of the program 
and the way resources were allocated to it. Because the program’s 
expenditures were growing faster than the Department’s overall 
budget, INAC had been reallocating funding from other programs. 

In our 2008 audit, we also noted that INAC had joined with the 
Government of Alberta and First Nations in that province to 
introduce a new child and family services program emphasizing 
prevention. This was a departure from the existing model, which 
focused on intervention with families and children at risk. 


4.48 In this follow-up audit, we assessed the progress made by INAC 
to implement two of our recommendations from our 2008 audit on the 
Department’s Child and Family Services Program. We also examined 
actions taken in relation to the government’s commitment in response 
to a recommendation made by the House of Commons Standing 
Committee on Public Accounts. 


4.49 Comparability of services. In our 2008 audit, we recommended 
that INAC define what is meant by its policy requirement that 
services be reasonably comparable, define its expectations for culturally 
appropriate services, and implement these into its program. INAC 
agreed with the recommendation and committed to a clearer definition, 
in tripartite agreements, of comparability of services with those 
provided by provinces. We found that INAC has not defined what is 
meant by comparability. Until it does so, it is unclear what is the service 
standard for which the Department is providing funding and what level 
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of services First Nations communities can eventually expect to receive. 
We also found that the Department had not conducted a review of all 
social services available in the provinces to see whether they are the 
same as what is available to children on reserves. 


4.50 In our 2008 audit, we noted that INAC had entered into a new 
funding arrangement in Alberta, enabling First Nations child and family 
service agencies in that province to deliver services that comply with 
provincial legislation. However, we noted that the funding model under 
the new Alberta tripartite agreement did not address all of the funding 
disparities we had identified. Since that audit, INAC has expanded its 
Enhanced Prevention Focused Approach and has negotiated new 
tripartite framework agreements with five other provinces: Manitoba, 
Nova Scotia, Prince Edward Island, Quebec, and Saskatchewan. The 
new approach includes greater emphasis on prevention services and is 
intended to increase comparability with services offered in provinces. 
INAC officials expect that the approach will reduce the number of 
children in care, but it is too early to observe results. 


4.51 The new tripartite agreements enable the provision of additional 
services beyond those offered in INAC’s initial program. However, 
without having defined what is meant by comparability, the Department 
has been unable to demonstrate that its new Enhanced Prevention 
Focused Approach provides services to children and families living on 
reserves that are reasonably comparable to provincial services. 


4.52 We also found that the Department has not developed a 

formal definition of what it means by “culturally appropriate services.” 
In response to this recommendation, however, INAC has developed a 
guiding principle on what culturally appropriate services would entail. 
This principle has been embedded in the tripartite agreements with the 
provinces under the new approach. It is also reflected in the business 
plans of First Nations child and family service agencies. According to 
INAC, a guiding principle instead of a definition allows service 
providers to more readily adapt their programs to the culture in 

each community. 


4.53 Cost determination. In 2008, we noted that the Department 
was regularly using funding budgeted for other programs to meet its 
obligations for its First Nations Child and Family Services Program. 
We also noted that the Department required a 74 percent increase in 
its operating and prevention services budget for Alberta to meet the 
requirements of the new tripartite agreement. We recommended that 
INAC determine the full costs of meeting the policy requirements of 
the program. The Department agreed to regularly update its estimate 
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of the cost of delivering the program with the new approach on 
a province-by-province basis and to periodically review the 
program budget. 


4.54 In this audit, we found that the Department had identified the 
costs it would have to pay for services in each province before moving 


‘to the new Enhanced Prevention Focused Approach. For its operations 


and prevention services, INAC determined that it would require an 
incremental increase of between 50 percent and 100 percent in its 
funding for each of the provinces in which it has established tripartite 
agreements to date. With all cost components taken into 
consideration, on average, the agreements led to an increase of over 
40 percent in the cost of INAC’s Child and Family Services Program in 
the participating provinces. 


4.55 Rather than using funds budgeted for other programs, INAC 

has obtained increased funding to run its Child and Family Services 
Program. The program’s budget has increased by 32 percent from 

$417 million in the 2005-06 fiscal year to $550 million in 2009-10. This 
rise partly reflects the increased funding levels needed to implement the 
new framework agreements. The Department also regularly conducts 
budget reviews and has taken steps to minimize the financial impact of 
its Child and Family Services Program on other INAC programs. 


4.56 Cost comparisons. The House of Commons Standing Committee 
on Public Accounts asked the Department for a comprehensive 
comparison of its funding to First Nations child and family welfare 
service agencies with provincial funding to similar agencies. The 
government committed to providing the Committee with a comparison 
of its funding for salaries and caseloads for the provinces that have made 
the transition to the new Enhanced Prevention Focused Approach. 


4.57 In preparation for framework negotiations with the provinces, 
INAC compared some elements central to the operations of child and 
family services programs with those of the provinces, such as social 
workers’ salaries and benefits. With this information, INAC provided to 
the Public Accounts Committee a comparison of some of its costs with 
those of the provinces that have established an agreement under the 
new approach. We note, however, that the Department has not provided 
information about social workers’ caseloads to the Committee because 
this is not public information. Moreover, certain services provided by 
provinces are outside INAC’s mandate—for example, services related 
to health issues and youth justice. The Department therefore does not 
provide these services. Our progress rating regarding child and family 
services is found in Exhibit 4.6. 
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Exhibit 4.6 Progress on addressing two recommendations on child and family services 


Indian and Northern Affairs Canada, in cooperation with provinces 
and First Nations agencies, should 


* define what is meant by services that are reasonably 
comparable, 


¢ define its expectations for culturally appropriate services and 
standards, and 


Unsatisfactory 
¢ implement this definition and these expectations into the 
program. 


(Recommendation 4.26 of the 2008 May Report of the Auditor 
General of Canada, Chapter 4, First Nations Child and Family 
Services Program—Indian and Northern Affairs Canada) 


Indian and Northern Affairs Canada should determine the full costs 
of meeting the policy requirements of the First Nations Child and 
Family Services Program. It should periodically review the 
program's budget to ensure that it continues to meet program 
requirements and to minimize the program’s financial impact on 
other departmental programs. 


(Recommendation 4.74 of the 2008 May Report of the Auditor 
General of Canada, Chapter 4) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


INAC has improved processes for implementing comprehensive land claim 
agreements but it is too early to identify impacts 


4.58 Comprehensive land claim agreements are intended to provide 
clarity and certainty with respect to ownership and use of land and 
resources, and to put in place structures for managing development. 
Negotiating these agreements is a major undertaking that can take 
decades. Managing the agreements afterwards requires ongoing 
commitment and vigilance by all parties. INAC is responsible for 
managing the federal responsibilities set out in the agreements and for 
monitoring the results achieved. In past audits, we have noted that 
INAC was not committed to the spirit and intent of the agreements 
and sometimes did not fulfill Canada’s obligations under them. In this 
audit, we found that INAC has taken steps toward meeting the spirit 
and intent of comprehensive land claim agreements, although more 
attention is still required. 


4.59 In 2003, we audited the federal implementation of 
comprehensive land claim agreements with the Gwich’in in the 
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Northwest Territories and the Inuit of Nunavut. In 2007, we audited 
the federal government’s implementation of its obligations related to 
the Inuvialuit Final Agreement. In these audits, we found that INAC 
and other federal departments were not implementing some key 
federal obligations. Moreover, we found that departments were 
considering only the letter of their land claim implementation 
responsibilities, not the spirit or intent of the agreements. In 2007, 
INAC officials described the principles of the Inuvialuit Final 
Agreement as being only Inuvialuit goals, not something to which the 
Government of Canada adheres. We also found that INAC was not 
coordinating activities to implement federal land claim obligations that 
were the responsibility of other departments, notably obligations 
related to contracting provisions. 


4.60 In this follow-up audit, we assessed the progress made by INAC 
on four previous recommendations that appeared in two audits we 
completed in 2003 and 2007 concerning the Nunavut, Gwich’in, 

and Inuvialuit comprehensive land claim agreements. We reviewed 
departmental documentation and interviewed INAC officials and 
claimant group representatives. We also reviewed and tested a new 
departmental database for tracking land claim obligations. 


4.61 Spirit and intent. In 2003, we noted that INAC was not 
considering the spirit and intent of the comprehensive land claim 
agreements we examined. We recommended that INAC focus not only 
on obligations but also on results. In this audit, we found that INAC 
is beginning to take into account the spirit and intent of 
comprehensive land claim agreements. This was particularly evident 
in the Department’s actions to monitor and achieve the objectives of 
agreements. Since our 2003 and 2007 audits, INAC has developed 
guidance indicating that a broader interpretation of the Government 
of Canada’s obligations under comprehensive land claim agreements 
is appropriate. The guidance states that since the Government of 
Canada has signed land claim agreements, it must work with other 
signatories to fulfill federal obligations and make progress toward 
mutually shared goals. INAC implementation officials have also 
indicated the importance of focusing on the treaty relationship to 
achieve progress toward the common goals of the agreements. For 
example, after the completion of a study on economic opportunities 
with Inuvialuit communities, INAC is now working with the Inuvialuit 
to explore other means of achieving the economic objectives of the 
Inuvialuit Final Agreement. 


4.62 We noted that the Deputy Minister had written to other 
departments to inform them of the approach to implementing modern 


Status Report of the Auditor General of Canada—2011 


Chapter 4 27 


PROGRAMS FOR FIRST NATIONS ON RESERVES 


28 Chapter 4 


treaties and INAC’s objective of improving interdepartmental 
coordination. However, at the time of our audit, INAC was still 
finalizing this guidance and had not communicated it formally to 
officials in other departments with responsibilities for implementing 
land claim agreements; nor had it developed any training for federal 
officials to ensure consistent implementation of the new approach. 
INAC had also not formally communicated the new approach to 
claimant groups. Some claimant group representatives told us that 
they have yet to see a significant change in the Department’s 
willingness to work toward achieving the objectives of comprehensive 
land claim agreements. 


4.63 Measuring performance. In our 2003 audit, we recommended 
that INAC provide measurable milestones and targets to gauge 
progress. Similarly, in our 2007 audit, we recommended that INAC 
develop performance indicators to measure progress toward meeting 
the principles of the Inuvialuit Final Agreement, and that it publicly 
monitor and report progress achieved. The Department undertook to 
propose performance indicators at a future implementation committee 
meeting, and to begin monitoring and reporting in spring 2008. 


4.64 We found that INAC has developed performance indicators 

to measure progress toward meeting the broader objectives of 
comprehensive land claim agreements. The Department has also 
conducted an evaluation of results achieved through the agreements. 
The evaluation reviewed progress realized toward six broad areas of 
focus from four agreements, including the Inuvialuit Final Agreement. 
It did not assess achievement of objectives of individual agreements but 
instead considered the impact that agreements are having overall. 

For example, the evaluation concluded that comprehensive land claim 
agreements have made an important contribution to Aboriginal business 
development and that regulatory regimes established through the 
agreements have led to the development of collaborative, consensus- 
based decision-making processes. INAC officials told us that they plan 
to evaluate the impacts of land claim agreements every five years. 


4.65 Coordinating federal obligations. In our 2003 audit, we 
recommended that INAC strengthen its coordinating framework to 
ensure that the Government of Canada meets federal responsibilities 
under land claim agreements. The Department undertook to continue 
working with an interdepartmental steering committee to ensure that 
land claim implementation issues are brought to the attention of senior 
officials in federal departments. We found that INAC has since led the 
development of an implementation management framework for 
comprehensive land claim agreements. The framework is applied by 
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regional and federal working groups of officials from various 
departments, as well as the interdepartmental steering committee of 
senior management chaired by INAC. These structures are intended 
to facilitate interaction between federal officials at the regional, 
headquarters, and senior management levels. The responsibilities of 
the working groups include 


_ © providing guidance to federal officials and helping them in their 
implementation roles, 


* coordinating activities between and within departments, and 


* monitoring and reporting on the implementation of federal 
obligations. 


We noted, however, that some tools to support the framework are not 
yet complete. 


4.66 We also found that improvements have been made to the 
process for implementing the federal contracting provisions of 
comprehensive land claim agreements, an area of concern identified 
in previous audits. In 2008, the Treasury Board of Canada revised its 
policy to clarify departments’ obligations when contracting within 
land claim jurisdictions. With this guidance, INAC now leads public 
reporting of federal procurement activities in land claim settlement 
areas across the country. We found that the Department has also 
undertaken initiatives to better educate other departments on 
contracting provisions within land claim jurisdictions. The Canada 
School of Public Service has developed a training course on 
procurement within land claim areas, with enrolment open to all 
procurement officers across the federal government. 


4.67 Implementing obligations. In 2007, we recommended that 
INAC identify each of Canada’s obligations under the Inuvialuit Final 
Agreement, identify the appropriate federal organizations to address 
them and clearly communicate the obligations to these organizations, 
develop a plan to implement the obligations, and regularly monitor 
and report to other signatories about Canada’s fulfillment of its 
obligations. INAC undertook to develop a strategy for communicating 
obligations to other federal organizations by March 2008, and to 
monitor and report on progress in fulfilling Canada’s obligations to 
other signatories of the agreement. 


4.68 To monitor progress toward fulfilling Canada’s obligations under 
comprehensive land claim agreements, INAC developed a new 
electronic data capture system in 2010. The system is designed to 
monitor the status of all federal government land claim obligations. 
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We randomly selected 10 obligations from two land claim agreements 
to determine the information about them captured in the database. 
We found that the database succeeded in capturing most of the 
obligations we examined, along with their status. At the time of our 
audit, however, the Department was still finalizing data accuracy and 
some of the database’s reporting functions. 


4.69 We randomly selected five federal obligations from each of the 
Gwich’in and Inuvialuit land claim agreements involving other 
departments’ responsibilities; we sought to determine whether INAC 
had clearly communicated to those departments the responsibilities of 
each. We found that INAC had not communicated these specific 
obligations to other federal departments. It did not have a plan in place 
to ensure the fulfillment of their obligations under the agreements, and 
it had not monitored whether the departments had fulfilled their 
obligations. 


470 It is important to meet Canada’s obligations under land claim 
agreements and to monitor progress toward the agreements’ objectives. 
Doing so will ensure that claimant groups receive the benefits agreed 
to under the agreements. Since 2007, INAC has made progress in 
providing guidance to other departments about the implementation of 
land claim agreements and has begun to evaluate their results. The 
Department has also begun to monitor progress toward fulfilling 
specific federal obligations (Exhibit 4.7). Continuing cooperation will 
be required to maintain progress in this area and realize the socio- 
economic and community improvements that comprehensive land 
claim agreements are intended to provide. 


INAC’s streamlining efforts have not lessened the burden of reporting requirements 
for First Nations 


471 The federal government has established many programs and 
services for First Nations communities. Most of them are delivered 
through funding arrangements such as contribution agreements that 
require the submission of reports to obtain funding and to account 
for funds used. Reporting requirements established by federal 
organizations can be a significant burden, especially for the many 
communities that have fewer than 500 residents. INAC and the 
Treasury Board of Canada Secretariat began efforts to streamline 
reporting by First Nations communities but the efforts have not 
been sustained. 
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Exhibit 4.7 Progress on addressing four recommendations on comprehensive land claim agreements 


Indian and Northern Affairs Canada should amend the land claim 
obligation system (LCOS) database to ensure that it focuses not 
only on obligations but also on results and that it provides 
measurable milestones and targets to gauge progress. 


(Recommendation 8.63 of the 2003 November Report of the 
Auditor General of Canada, Chapter 8, Indian and Northern Affairs 
Canada—Transferring Federal Responsibilities to the North) 


Satisfactory 


In cooperation with the Inuvialuit, and with the Yukon and 
Northwest Territories governments, Indian and Northern Affairs 
Canada should develop performance indicators to measure 
progress towards meeting the principles of the /nuvia/uit Final 
Agreement, and should publicly monitor and report progress to 
other signatories. 


(Recommendation 3.85 of the 2007 October Report of the Auditor 
General, Chapter 3, Inuvialuit Final Agreement) 


Satisfactory 


Indian and Northern Affairs Canada should strengthen its 
coordinating framework to ensure that the Government of Canada 
meets federal responsibilities under the land claims agreements. 


(Recommendation 8.77 of the 2003 November Report of the 
Auditor General of Canada, Chapter 8) 


Satisfactory 


Indian and Northern Affairs Canada should develop a strategic 
approach towards implementing Canada’s obligations under the 
Inuvialuit Final Agreement. Such an approach should, at a 
minimum 


identify each of Canada’s obligations and the appropriate 
federal organizations to address them, and should clearly 
communicate their obligations to these federal organizations; 


develop a plan to implement federal obligations; and 


regularly monitor and report to other signatories Canada’s 
fulfillment of its obligations. 


(Recommendation 3.76 of the 2007 October Report of the Auditor 
General of Canada, Chapter 3) 


Unsatisfactory 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 


and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 


and the time that has elapsed since the recommendation was made. 


4.72 In 2002, we looked at the amount of reporting required of First 
Nations by federal organizations. We estimated that four federal 
organizations together required about 168 reports annually from each 
First Nations reserve. We found that many of the reports were 
unnecessary and were not in fact used by the federal organizations. 

We followed up on this issue in 2006. At that time, we found that federal 
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departments had made little progress on meeting our recommendations 
to reduce reporting requirements. In our 2006 follow-up audit, 

we reported that INAC’s officials told us that the Department 
obtained more than 60,000 reports a year from over 600 First Nations 
communities. The Treasury Board of Canada Secretariat analyzed the 
extent of federal involvement with First Nations and confirmed the 
seriousness of the problem we had identified in 2002. 


473 In this follow-up audit, we reviewed the steps taken by the 
Treasury Board of Canada Secretariat and INAC to streamline 
reporting requirements for recipients of federal funding, including First 
Nations communities. We also assessed the progress made to 
implement one of our 2002 recommendations. 


4.74 Streamlining reporting throughout the government. In 

June 2006, the President of the Treasury Board commissioned an 
independent panel to review and recommend ways of simplifying the 
administration of grants and contributions. Later that year, the panel 
identified a need for the federal government to dramatically simplify its 
reporting and accountability regime related to grant and contribution 
agreements. It recommended that the Treasury Board and departments 
modify their monitoring and recipient reporting requirements to avoid 
duplication or redundancy, and ensure that requirements are clearly 
connected to a demonstrable need. 


4.75 The Treasury Board of Canada Secretariat responded in 2008 

by issuing the Government of Canada Action Plan to Reform the 
Administration of Grant and Contribution Programs. This commits 
the government to reducing recipients’ administrative and reporting 
burden. The Secretariat created a new centre of expertise to lead and 
coordinate the implementation of the action plan. One of its elements 
was a Call for departments to develop an action plan to improve the 
delivery and efficiency of grants and contributions. Another element 
was the issuing of the revised Policy on Transfer Payments and new 
Directive, both of which came into effect in 2008. Under the revised 
policy, deputy heads are responsible for ensuring that the 
administrative requirements for recipients are proportionate to the risk 
level. Departments should determine reporting requirements after 
performing a risk assessment. Departments needed to have a system for 
conducting risk assessments in place by March 2010. INAC officials 
told us that the Department received verbal approval to extend this 
date to 2011. The Secretariat does not monitor efforts to reduce the 
reporting burden since it considers this to be the responsibility of 
deputy heads as the chief accounting officers of their departments. 
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476 We found that at the time of our audit, INAC had yet to finalize 
a process for assessing risks associated with contribution agreements. 
Consequently, the Department has yet to determine the level of 
reporting requirements most appropriate to each First Nation, as 
required under the 2008 Transfer Payment Policy. 


477 INAC reporting initiatives. In 2002, we recommended that 

the federal government regularly review its reporting requirements, 
determine reporting needs when new programs are set up, and drop 
unnecessary or duplicative reporting. The government agreed that 
reporting should be administratively efficient and rationalized across the 
federal government. It committed to reviewing reporting requirements 
individually at the departmental level and collectively as a government- 
wide initiative. We found that INAC started data collection review 
efforts in 2003, but these ended in 2007. The aim of the review was to 
understand the rationale for reporting requirements, including the 
extent of the reporting burden on First Nations. The efforts resulted in 
the creation of the Data Collection Policy in 2004. 


4.78 In this audit, we found that INAC sought to better understand 
its estimate of 60,000 reports required yearly from First Nations. 

To arrive at this estimate, INAC had counted all reports under all of 
its previous regional data systems related in some way to grants and 
contributions. The total included many reports not directly required 
under contribution agreements, such as a requirement to give INAC 
updates about all changes to the First Nations population on each 
reserve, including births, deaths, and moves. 


479 Continuation of other initiatives. In 2007, INAC started the 
Smart Reporting Initiative to focus on performance measurement, with 
the aim of determining what information was actually required for 
management and accountability purposes. In June 2010, the Deputy 
Minister named a special representative to lead another initiative to 
reduce reporting requirements. 


4.80 Electronic system to streamline reporting. In 2006, INAC 
implemented an electronic system for the management of transfer 
payments: the First Nations and Inuit Transfer Payment System 
(FNITP). The system was designed partly to reduce the reporting 
burden by allowing First Nations to submit their reports electronically 
and allowing the Department to streamline administrative processes. 


4.81 We found that the system has led to a streamlining of some 
reporting requirements. It ensures that reporting requirements related 
to funding agreements include only those stipulated in the provisions 
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of the agreements. The Department was able to identify several reports 
that were eliminated and others that were reduced, a result officials 
attributed partly to the introduction of the FNITP For example, 
INAC’s National Reporting Guide for 2009-10 notes that INAC has 
eliminated 9 national reports, 7 regional reports, and fields in 12 other 
reports. Within the system, there are also procedures in place for 
capturing and managing information on funding agreements to ensure 
compliance with financial authorities. 


482 INAC provided some training to First Nations on using the 
FNITP. but many cannot take advantage of the system because of 
limited capacity and high staff turnover. At the time of our audit, 

334 of approximately 700 First Nations and tribal councils had access 
to the FNITP; 228 of these were regularly using the system. For First 
Nations and tribal councils without access to the FNITR INAC 
continues to receive reports separately and enters the information into 
the system manually. 


483 Recently increased report requirements. Despite many 
initiatives, we have not seen a significant reduction in the reporting 
burden. We were able to track the number of reports actually required 
from First Nations since the introduction of the FNITP because it stores 
all reporting related to contribution agreements. The number of 
required reports increased from 30,000 in the 2007-08 fiscal year 

to 32,000 in 2009-10. The increase may be attributable to new 
accountability demands across government and new programs available 
to First Nations. First Nations officials with whom we spoke also told us 
that they had not seen a reduction in reporting requirements since our 
last audit, and many indicated that the reporting burden has increased 
in recent years. 


4.84 INAC developed a Data Collection Policy in 2004 and drafted 
related guidance in 2007. The policy called for reporting requirements to 
be assessed before programs were renewed or new programs were set up. 
We examined the program approval process for six programs developed | 
or renewed between April 2007 and March 2010 to determine whether 
this assessment had been conducted. We found that the policy had not 
been applied in the majority of these programs, and reporting 
requirements were not reviewed when programs were set up or renewed. 


4.85 As we mentioned in the preface, we remain concerned about the 
burden associated with the federal reporting requirements, particularly 
related to INAC’s contribution agreements with First Nations. Many 
initiatives with the potential to streamline reporting have been started 
but have not resulted in meaningful improvement. INAC does not 
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apply the Data Collection Policy and has not conducted all of the risk 
assessments of First Nations recipients, as required by the new federal 
Transfer Payment Policy. Thus, it remains unclear whether this degree 
of reporting helps make First Nations accountable, or whether it assists 
either the Department or First Nations with their management 
responsibilities. First Nations continue to spend time and resources to 
complete reports for INAC, but some of these reports may serve little 
purpose and may interfere with First Nations’ ability to meet the needs 
of their members. Our progress rating regarding reporting requirements 


is found in Exhibit 4.8. 


Exhibit 4.8 Progress on addressing a recommendation on reporting requirements 


The fed 
reporting requirements on a regular basis and to determine 
reporting needs when new programs are set up. Unnecessary or 
duplicative reporting requirements should be dropped. Unsatisfactory 


(Recommendation 1.61 of the 2002 December Report of the 
Auditor General of Canada, Chapter 1, Streamlining First Nations 
Reporting to Federal Organizations) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


486 Recommendation. Indian and Northern Affairs Canada, in 
consultation with relevant federal organizations, should develop 
implementation plans for the recommendations on which progress to 
date has been unsatisfactory or incomplete. The plans should include 
specific goals, targets, action items, timelines for achieving results, and 
indicators for measuring progress. 


The Department’s response. Agreed. INAC agrees to develop 
implementation plans as recommended, with the following 
understandings in relation to specific areas: 


¢ The First Nation Student Success Program and the Education 
Partnerships Program are part of the Reforming First Nation 
Education Initiative that was launched in 2008 to set the 
foundation for long-term improvement in education outcomes. 
Tripartite agreements are in place in seven provinces, including a 
sub-regional agreement with Saskatchewan and an 
implementation-ready tripartite K-12 partnership in 


British Columbia. 
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¢ INAC supports Health Canada’s efforts respecting drinking water 


and cooperates fully. 


A strategy to address the problem of mould on reserves is being 
advanced through the existing Indoor Air Quality Committee 
with representatives from the Canada Mortgage and Housing 
Corporation, Health Canada, and the Assembly of First Nations, 
as well as through enhanced funding as opportunities arise. 


On child and family services, considerable progress has been made 
on defining comparability while respecting differences between 
jurisdictions. The guiding principle that First Nations themselves 
are best suited to define culturally appropriate services is now 
incorporated in all tripartite discussions, as is the matter of 
culturally appropriate placement options. 


In relation to the implementation and monitoring of the 
Inuvialuit Final Agreement and other agreements, progress is 
being made through the development of the Treaty Obligation 
Monitoring System (TOMS). In the last year, TOMS was 
presented to other government departments through regional 
caucuses in the Atlantic Region, Quebec, and the 

Northwest Territories, and plans are under way for similar 
presentations across the rest of the country. TOMS will be made 
accessible online to other departments over the next year. In 
addition, the Deputy Minister has communicated to colleagues 
the importance of their active engagement in implementation. 
Implementation working groups support efforts by colleagues in 
other departments to collaborate on implementation at the 
national or regional level. Guidelines are being developed to 
provide information and direction on critical issues such as 
alternative dispute resolution. 


In relation to reporting requirements, consultations with First 
Nations and all stakeholders regarding this matter are ongoing 
through a recently created senior management position, reporting 
directly to the Deputy Minister. 


Conclusion 


487 The federal government supports many services to First Nations 
that provinces and municipalities provide to people off reserves. In 
reports to Parliament during the past decade, the Office of the Auditor 
General published 16 chapters directly addressing First Nations and 
Inuit issues, and made 91 recommendations to Indian and Northern 
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Affairs Canada (INAC) and other federal organizations to address a 
wide range of issues. This audit followed up on 12 of the 
recommendations. 


4.88 We found that INAC has made satisfactory progress in 
implementing some of our recommendations. The Department has 
demonstrated greater commitment to meeting the spirit and intent of 
land claim agreements and measuring their impacts. INAC has also 
strengthéned its coordinating framework so that other federal 
organizations understand their obligations under the agreements. 
INAC has assessed the costs of delivering education services on 
reserves and compared these with provincial costs. INAC and Health 
Canada have drafted legislation to provide a statutory basis for 
ensuring safe drinking water on reserves, but it may be years before 
regulations are in place. 


4.89 Notwithstanding all the actions taken and efforts made, we 
found that INAC, the Canada Mortgage and Housing Corporation, 
and Health Canada have not made satisfactory progress in 
implementing several of our recommendations. The recommendations 
relate to some of the most important issues of concern to First Nations, 
including education, housing, child and family services, and 
administrative reporting requirements. The three federal organizations 
have made repeated commitments to action. Nevertheless, we found 
that those commitments and subsequent actions have often not 
resulted in improvements. In some cases, conditions have worsened 
since our earlier audits: the education gap has widened, the shortage of 
adequate housing on reserves has become more acute, and 
administrative reporting requirements have become more onerous. 


4.90 Although the federal government has taken action on many of 
the issues covered in this audit, we note that it has not been an easy 
task to address these complex issues. Further, conditions on reserves 
have not improved much in the last decade. We believe that structural 
problems have inhibited greater progress toward improving the lives 
and well-being of those living in First Nations reserves. In the preface 
of this chapter, we discuss some of the problems. 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these 
standards as the minimum requirement for our audits, we also draw upon the standards and practices of 
other disciplines. 


Objectives 


The overall audit objective was to determine whether Indian and Northern Affairs Canada (INAC), 
Health Canada, the Canada Mortgage and Housing Corporation (CMHC), and the Treasury Board of 
Canada Secretariat have made satisfactory progress in addressing key observations and implementing key 
recommendations on First Nations programs from previous chapters related to Aboriginal issues appearing 
in reports of the Office of the Auditor General. 


The sub-objectives were to determine whether 


¢ Indian and Northern Affairs Canada has made satisfactory progress in addressing key observations and 
implementing key recommendations on First Nations education programs from our 
2004 November Report, Chapter 5, Indian and Northern Affairs Canada—Education Program and 
Post-Secondary Student Support; 


Indian and Northern Affairs Canada has made satisfactory progress in addressing key observations and 
implementing key recommendations from our 2008 May Report, Chapter 4, First Nations Child and 
Family Services Program—Indian and Northern Affairs Canada; 


Indian and Northern Affairs Canada, Health Canada, and the Canada Mortgage and Housing 
Corporation have made satisfactory progress in addressing key observations and implementing key 
recommendations from our 2003 April Report, Chapter 6, Federal Government Support to First 
Nations—Housing on Reserves; 


Indian and Northern Affairs Canada and Health Canada have made satisfactory progress in addressing 
key observations and implementing key recommendations from the 2005 September Report of the 
Commissioner of the Environment and Sustainable Development, Chapter 5, Drinking Water in First 
Nations Communities; 


Indian and Northern Affairs Canada and the Treasury Board of Canada Secretariat have made 
satisfactory progress in addressing key observations and implementing key recommendations from our 
2002 December Report, Chapter 1, Streamlining First Nations Reporting to Federal Organizations; 
and 


Indian and Northern Affairs Canada has made satisfactory progress in addressing key observations and 
implementing key recommendations on First Nations land claims from our 2003 November Report, 
Chapter 8, Indian and Northern Affairs Canada—Transferring Federal Responsibilities to the North, 
and our 2007 October Report, Chapter 3, Inuvialuit Final Agreement. 
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Scope and approach 


The entities included in this follow-up audit are Indian and Northern Affairs Canada, Health Canada, the 
Canada Mortgage and Housing Corporation, and the Treasury Board of Canada Secretariat. INAC is the 
federal department primarily responsible for meeting the Government of Canada’s obligations and 
commitments to First Nations, and was included in all the audit lines of enquiry. Health Canada was 
included for its role in drinking water on reserves and in addressing health considerations in housing on 
reserves, such as mould. The Canada Mortgage and Housing Corporation was included for its role in 
addressing the housing shortage, improving housing conditions, and addressing the mould issue in 
on-reserve housing. Since past recommendations about streamlining reporting had been addressed to the 
federal government, the Treasury Board of Canada Secretariat was also part of the audit. We obtained and 
reviewed related documentation and interviewed officials from the four entities to assess progress in 
implementing past recommendations. 


For the representative sample in the water section, we selected the three regions with the most water 
advisories and randomly sampled within those regions. Our examination of land claim obligations relied 
on random selections. 


While First Nations were not subject to audit, we visited selected First Nations reserves and organizations 
to obtain their perspectives on progress made since we conducted our audits. 


Criteria 


Education 


Indian and Northern Affairs Canada (INAC), in consultation with | * 2004 November Report of the Auditor General of Canada, 
First Nations, has developed and implements a strategy and Chapter 5, recommendation 5.33 
action plan, with targets, to close the education gap. It also 


d ; : ; ¢ INAC’s response to 2004 November recommendation 5.33 
reports progress to Parliament and to First Nations on a timely 


basis. * Education Action Plan, INAC, 2005 

¢ Report on Plans and Priorities, 2009-10, INAC 
INAC obtains information on the costs of delivering education * 2004 November Report of the Auditor General of Canada, 
services on reserves and compares the costs with those of Chapter 5, recommendation 5.51 


providing comparable education services in the provinces. * INAC's response to 2004 November recommendation 5.51 


Education Action Plan, INAC, 2005 


INAC reviews its post-secondary funding mechanisms, including June 2005 Report of the Standing Committee on Public 
the use of flexible transfers that are designed to deliver equitable Accounts (PAC), recommendation 11 (related to Auditor 
funding to as many eligible students as possible. General’s 2004 November Report, Chapter 5, 
recommendation 5.94 


¢ Government response to the 2005 PAC Report 


Action Plan for the 2009 Internal Audit on the Post-Secondary 
Education Program, INAC 
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Child and family services 


INAC, in cooperation with provinces and First Nations agencies, 


defines what is meant by services that are reasonably 
comparable, 


defines its expectations for culturally appropriate services and 
standards, and 


implements this definition and these expectations into the 
program. 


INAC determines the costs of meeting the policy requirements of 
the First Nations Child and Family Services Program. It then 
periodically reviews the program’s budget to ensure that it 
continues to meet program requirements and minimizes the 
program's financial impact on other departmental programs. 


* 2008 May Report of the Auditor General of Canada, 
Chapter 4, recommendation 4.26 


* INAC’s response to 2008 May recommendation 4.26 


* National Program Manual, First Nations Child and Family 
Services 


* 2008 May Report of the Auditor General of Canada, 
Chapter 4, recommendation 4.74 


* INAC’s response to 2008 May recommendation 4.74 


* Policy on Management, Resources and Results Structure 
(sections 5.2.1 and 5.2.2), Treasury Board, 2010 


* National Program Manual, First Nations Child and Family 
Services 


INAC conducts a comparison of its funding to First Nations child 
and family welfare services agencies to provincial funding of 
similar agencies. 


* March 2009 Report of the Standing Committee on Public 
Accounts, recommendation 2; 2008 May Report of the 
Auditor General, Chapter 4, recommendations 4.26 and 4.74 


* Government response to PAC Report 


* National Program Manual, First Nations Child and Family 
Services 


Housing 


INAC and Canada Mortgage and Housing Corporation (CMHC) 
take action to reduce the housing shortage that existed in 2003. 


* 2003 April Report of the Auditor General of Canada, 
Chapter 6, paragraph 6.1 


¢ Report on Plans and Priorities, 2007-08, INAC 
* Budget 2005 


INAC and CMHC take action to reduce the percentage of houses 
requiring renovations. 


2003 April Report of the Auditor General of Cantecls, 
Chapter 6, paragraph 6.1 


Report on Plans and Priorities, 2007-08, INAC 
¢ Budget 2005 


INAC, CMHC, and Health Canada, in consultation with First 
Nations, develop a strategy and action plan to address the 
problem of mould on reserves. 


‘|e 


2003 April Report of the Auditor General of Canada, 
Chapter 6, recommendation 6.48 


Responses from the three organizations te 2003 April 
recommendation 6.48 


* Terms of Reference of the Interdepartmental Committee on 


i Mould 
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Drinking water 


INAC and Health Canada, in consultation with First Nations, 
develop and implement a regulatory regime for drinking water in 
First Nations communities. This regime is comparable with that 
in provinces and designed to protect the health and safety of First 
Nations people. As a minimum, this regime deals with roles and 
responsibilities, water quality requirements, technical 
requirements, certification of systems and operators, compliance 
and enforcement, and public reporting requirements. 


2005 September Report of the Commissioner of the 
Environment and Sustainable Development, Chapter 5, 
recommendation 5.36 


Departments’ responses to 2005 September recommendation 
5.36 


2010 Speech from the Throne 


Health Canada, in consultation with INAC and First Nations, 
ensures that 


drinking water tests recommended under the Guidelines for 
Canadian Drinking Water Quality are being carried out; 


test results are properly recorded and relevant information is 
shared with appropriate parties; 


it identifies and defines actions to be taken in situations where 
drinking water is not safe, or where there is no assurance of 
drinking water safety; and 


* actions are taken by responsible parties as required. 


2005 September Report of the Commissioner of the 
Environment and Sustainable Development, Chapter 5, 
recommendation 5.69 


Departments’ responses to 2005 September recommendation 
5.69 


Procedure Manual for Safe Drinking Water in First Nations 
Communities South of 60°, Health Canada. 


Protocol for Safe Drinking Water in First Nations Communities, 
INAC 


INAC reduces the number of drinking water systems in First 
Nations communities posing a significant risk to the quality or 
the safety of drinking water. 


2005 September Report of the Commissioner of the 
Environment and Sustainable Development, Chapter 5, 
paragraph 5.13 


Report on Plans and Priorities, 2006-07, INAC, page 31 


Reporting requirements 


INAC and the Treasury Board of Canada Secretariat review 
reporting requirements of contribution programs and eliminate 
unnecessary or duplicative reporting requirements. 


2002 December Report of the Auditor General of Canada, 
Chapter 1, recommendation 1.61 


Government’s response to 2002 December recommendation 
1.61 


Data Collection Policy, INAC, 2004 


Policy on Transfer Payments, sections 6.5.5 and 6.5.10, 
Treasury Board, 2008 


News release, 14 February 2007: Canada’s New Government 
Acting to Improve Grant and Contribution Programs, Treasury 
Board of Canada Secretariat 


INAC has systems and processes to capture, manage, and assess 
whether First Nations reports in a timely manner. 


Directive on Transfer Payments, section 6.5.2, Treasury Board, 
2008 


Directive on Recordkeeping, sections 5.1.1 and 6.1.3, 
Treasury Board, 2009 


Policy-on Information Management, section 5.2.3, Treasury 
Board, 2007 
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* 2002 December Report of the Auditor General of Canada, 
Chapter 1, recommendation 1.61 


INAC, in consultation with First Nations, reviews reporting 
requirements to determine reporting needs when new programs 
are set up and requires only information that is needed for 


om * Government's response to 2002 December recommendation 
management or accountability purposes. 


1.61 
* Data Collection Policy, INAC, 2004 


INAC has a system that monitors both land claim obligations and | * 2003 November Report of the Auditor General of Canada, 
land claim results. The system also provides measurable | Chapter 8, recommendation 8.63 
ee ee ee | « INAC’s response to 2003 November recommendation 8.63 
* Results for Canadians: A Management Framework for the 
Government of Canada, Treasury Board of Canada 
Secretariat, 2000 


INAC’s coordinating framework is designed to ensure that the de 2003 November Report of the Auditor General of Canada, 
Government of Canada meets federal responsibilities under land | Chapter 8, recommendation 8.77 
claims agreements. 


| ¢ INAC’s response to 2003 November recommendation 8.77 


* Implementation of comprehensive land claim and self- 
| government agreements: A handbook for the use of federal 
| Officials, INAC, 2003 

Ss =| 


INAC develops a strategic approach toward implementing 2007 October Report of the Auditor General of Canada, 
Canada’s obligations under the Inuvialuit Final Agreement. Such | Chapter 3, recommendation 3.76 
an approach, at a minimum, 


| « INAC's response to 2007 October recommendation 3.76 
¢ identifies each of Canada’s obligations and the appropriate y gies 
federal organizations to address them, and communicates INAC 7 mankate as lead federal organization for 
| implementation 


their obligations to these federal organizations; 
* Memoranda to Cabinet, 1984-85 


develops a plan to implement federal obligations; and 


Implementation of comprehensive land claim and self- 
government agreements: A handbook for the use of federal 
Officials, INAC, 2003 


monitors and reports to other signatories Canada’s fulfillment 
of its obligations. 


Results for Canadians: A Management Framework for the 
Government of Canada, Treasury Board of Canada 
Secretariat, 2000 


In cooperation with the Inuvialuit, and with the Yukon and 2007 October Report of the Auditor General of Canada, 
Northwest Territories governments, INAC develops performance | Chapter 3, recommendation 3.85 

indicators to measure progress toward meeting the principles of 

the Inuvialuit Final Agreement, and publicly monitors and reports 
progress to other signatories. 


« INAC’s response to 2007 October recommendation 3.85 
Memoranda to Cabinet, 1984 


Results-based Management and Accountability Framework, 
INAC, 2001 


¢ Comprehensive Land Claims Policy, INAC, 1986 


Management reviewed and accepted the suitability of the criteria used in the audit. 
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Period covered by the audit 


The audit covered the years 2002 to 2010. Audit work for this chapter was substantially completed on 
1 November 2010. 


Audit team 


Assistant Auditor General: Ronnie Campbell 
Principal: Frank Barrett 
Director: Nadine Cormier 


Irene Andayo 

Laurent Bergeron Collin 
Alexandre Boucher 
Maria Pooley 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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Appendix. List of recommendations 


The following is the recommendation found in Chapter 4. The number in front of the recommendation 
indicates the paragraph where it appears in the chapter. 


Agreed. INAC agrees to develop implementation plans as 
recommended, with the following understandings in relation to 
specific areas: 


4.86 Indian and Northern Affairs 
Canada, in consultation with relevant 
federal organizations, should develop 
implementation plans for the 
recommendations on which progress to 
date has been unsatisfactory or 
incomplete. The plans should include 
specific goals, targets, action items, 
timelines for achieving results, and 
indicators for measuring progress. 


The First Nation Student Success Program and the Education 
Partnerships Program are part of the Reforming First Nation 
Education Initiative that was launched in 2008 to set the 
foundation for long-term improvement in education outcomes. 
Tripartite agreements are in place in seven provinces, including 
a sub-regional agreement with Saskatchewan and an 
implementation-ready tripartite K-12 partnership in 

British Columbia. 


INAC supports Health Canada’s efforts respecting drinking 
water and cooperates fully. 


A strategy to address the problem of mould on reserves is being 
advanced through the existing Indoor Air Quality Committee 
with representatives from the Canada Mortgage and Housing 
Corporation, Health Canada, and the Assembly of First 
Nations, as well as through enhanced funding as opportunities 
arise. 


On child and family services, considerable progress has been 
made on defining comparability while respecting differences 
between jurisdictions. The guiding principle that First Nations 
themselves are best suited to define culturally appropriate 
services is now incorporated in all tripartite discussions, as is 
the matter of culturally appropriate placement options. 


In relation to the implementation and monitoring of the 
Inuvialuit Final Agreement and other agreements, progress is 
being made through the development of the Treaty Obligation 
Monitoring System (TOMS). In the last year, TOMS was 
presented to other government departments through regional 
caucuses in the Atlantic Region, Quebec, and the 

Northwest Territories, and plans are under way for similar 
presentations across the rest of the country. TOMS will be 
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made accessible online to other departments over the next 
year. In addition, the Deputy Minister has communicated to 
colleagues the importance of their active engagement in 
implementation. Implementation working groups support 
efforts by colleagues in other departments to collaborate on 
implementation at the national or regional level. Guidelines 

‘are being developed to provide information and direction on 
critical issues such as alternative dispute resolution. 


* In relation to reporting requirements, consultations with First 
Nations and all stakeholders regarding this matter are ongoing 
through a recently created senior management position, 
reporting directly to the Deputy Minister. 
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What we examined 


Why it’s important 


National Police Services— 
Royal Canadian Mounted Police 


Main Points 


The Royal Canadian Mounted Police (RCMP) provides Canada’s law 
enforcement community with specialized national police services such 
as forensic analyses of criminal evidence, criminal records information, 
identification services, technological support, learning opportunities, 
and the collection and analysis of criminal information and 
intelligence. 


This follow-up audit assessed the progress that the federal government 
and the RCMP have made toward keeping commitments to improve 
certain national police services, including their timeliness, made in 
response to our audits in 2000, 2004, and 2007; those audits noted 
significant delays in the delivery of these services. We focused on 

four of the national police services: Forensic Laboratory Services, the 
National DNA Data Bank, Criminal Intelligence Service Canada, and 
Canadian Criminal Real Time Identification Services. The DNA Data 
Bank has not been the subject of a previous audit by this office. 


We also followed up on previous audit observations concerning the 
governance of national police services as a whole and the approach to 
funding them. 


Audit work for this chapter was substantially completed 


on 1 November 2010. 


Jurisdictional limits on individual police forces, the division of 
investigative responsibility when crime crosses these jurisdictional 
lines, and constraints on funds and manpower limit the ability of any 
single police force to carry out complex investigations. Given the 
RCMP’s federal and contractual policing responsibilities, provincial 
and municipal police forces often rely on the RCMP to provide highly 
specialized police support services. The aim of providing national 
police services is to enable and sustain uniform access to information 
that supports public safety and the administration of justice to all 
Canadians, particularly where such services may not be within the 
resources of individual jurisdictions. 
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What we found 


* The federal government and the Royal Canadian Mounted Police 
have made unsatisfactory progress on commitments made in 
response to recommendations in our 2000, 2004, and 2007 reports 
regarding national police services. The RCMP is still providing 
national police services to other levels of government and law 
enforcement organizations without clarifying which services should 
be provided; how they should be funded; and how provincial, 
territorial, and municipal partners should be engaged within the 
national police services governance framework. In the absence of 
formal agreements, the accountability of the RCMP and other 
partners for the delivery of these services has not been clearly 
established, nor has how they are to be sustained over the long term. 


* The RCMP has not determined the detailed transactional costs of 
providing each of its national police services. Accurate costing 
information is essential to efficiently manage services and resources. 
In addition, because of financial pressures, the RCMP is finding it 
difficult to deliver national police services and also fulfill its other 
obligations. In light of existing resource constraints, it is difficult 
without adequate cost information to negotiate agreements on 
which national police services will be provided and which are 
priorities. 


While changes in the approach to managing Forensic Laboratory 
Services have increased the RCMP’s focus on clients’ needs and 
reduced turnaround times, other services have not shown similar 
improvement. For example, backlogs and delays in updating criminal 
record information have significantly increased—from 5 months 

in 2000 to 14 months today. In addition, technological upgrades to 
critical systems requiring significant investments are facing 
challenges or have been delayed. At the same time, the demand for 
national police services is growing and straining the RCMP’s ability 
to deliver in a timely way. 


The RCMP and Public Safety Canada have responded. The RCMP 
and Public Safety Canada agree with all of the recommendations. 
Their detailed responses follow the recommendations throughout the 
chapter. 
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NATIONAL POLICE SERVICES—ROYAL CANADIAN MOUNTED POLICE 


Introduction 


5.1. Under the Constitution, the provincial and federal governments 
share responsibility for the criminal justice system. Parliament has 
legislative authority for establishing criminal law and criminal 
procedures. Provinces have primary responsibility for enforcing the 
Criminal Code, prosecuting criminal charges, and administering the 
trial courts. The Royal Canadian Mounted Police (RCMP) contributes 
to national policing by providing three types of services. First, it 
provides federal policing by enforcing federal laws dealing with a 
variety of subjects such as organized crime and border security. Second, 
it provides contracted police services to all provinces and territories, 
except Quebec and Ontario, which each have their own provincial 
police force, and to certain municipalities and First Nations 
communities. Third, it provides support services to police forces across 
the country, such as forensic analyses of criminal evidence. These 
national support services for policing are the focus of this audit. 


§.2 The RCMP is part of the Public Safety Canada portfolio, 
reporting to the Minister. Public Safety Canada contributes to the 
development of law enforcement policies and programs and provides 
advice and support to the Minister in matters regarding the RCMP. 
Public Safety Canada also represents the federal government in 
negotiating police service agreements and leads collaborative efforts to 
modernize the management, governance, and review of the RCMP. 


5.3. The RCMP has a wide range of responsibilities. Through federal 
policing and through its various policing contracts with provinces and 
municipalities, the RCMP has jurisdictions and personnel that span all 
of Canada. This benefits all Canadians when dealing with crime that 
extends beyond any single jurisdiction or beyond the means of an 
individual police force. As we indicated in our November 2005 chapter 
on contract policing, these contracts enable the RCMP to act as a 
national police force, even though it is not mandated to do so, 
providing policing at all levels (municipal, provincial, national, 
international, and Aboriginal). With the approval of the federal 
government, the RCMP can provide specialized police support not 
only to the law enforcement community, but also to international 
events held in Canada, such as the Vancouver Olympics and the G-8 
and G-20 summits. 
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Overview of national police services 


5.4 The RCMP delivers a number of national police services— 
specialized investigational support services that assist Canada’s law 
enforcement community—such as 


* forensic analyses of criminal evidence, 
* criminal records information, 

* identification services, 

* technological support, 

¢ the Canadian Police College, and 


¢ the collection and analysis of criminal information and intelligence. 


5.5 The RCMP has been delivering such specialized services for 
more than a century. For example, it has provided fingerprint 
identification and centralized upkeep of criminal records since 1908 
and has provided criminal investigation support to the provinces 
since 1937. Although initially designed and funded to support RCMP 
members, these services were later made available to other forces and 
the criminal justice system in provinces and municipalities. 


5.6 In 1965, the federal and provincial governments agreed that the 
fight against the growing presence of organized crime would require 
increased cooperation among various jurisdictions as well as extensive 
additional specialized technological support and resources beyond the 
means of any one police force in Canada. Following a 1965 Cabinet 
decision and the 1966 Federal—Provincial Conference of Attorneys 
General on Organized Crime, the RCMP delivered new specialized 
services to support policing to Canada’s law enforcement community. 
These new services included the formation of Criminal Intelligence 
Service Canada, the Canadian Police College, and the Canadian Police 
Information Centre, as well as the expansion of the RCMP’s criminal 
record and fingerprint repositories and its Forensic Laboratory Services. 
These additions and expansions represented an initial federal 
government investment of $13.6 million over five years starting in 1966. 


5.7 Since 1966, the RCMP has also been tasked with additional 
specialized services, including 


¢ the Canadian Bomb Data Centre, 
* the Violent Crime Linkage Analysis System, 


¢ the National DNA Data Bank, 
* the Canadian Police Centre for Missing and Exploited Children, 
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* the National Sex Offender Registry, and 


¢ the Canadian Firearms Program. 


Exhibit 5.1 briefly describes these services. 


Exhibit 5.1 National police services provided over the years by the Royal Canadian Mounted Police 


Identification Services fingerprint and criminal record 
information. 
aie 
Forensic Laboratory Services Provides forensic analysis services TOS, 
relating to criminal cases in biology, 
toxicology, firearms, trace evidence, 
and counterfeit and document 
examinations. 
Canadian Police College Provides specialized and advanced 1966 
police sciences and leadership 
training to foreign and Canadian 
senior police officers. 
a a a t 
Canadian Police Information Serves as a central repository that 1966 
Centre provides information on crimes and 
criminals. 
Criminal Intelligence Service Facilitates the production and 1970 
Canada exchange of criminal information 
and intelligence. 
Canadian Bomb Data Centre Provides support and information in 1972 
combatting the illegal use of 
explosives. 
Violent Crime Linkage Serves as a central repository that 1991 
Analysis System provides information on serial 
criminals and violent crimes. 
National DNA Data Bank Serves as a central repository that 2000 
captures and references DNA 
profiles from crime scenes and 
convicted offenders. 
Canadian Police Centre for Provides investigative assistance to 2004 
Missing and Exploited help reduce the vulnerability and 
Children exploitation of children. 
4 
National Sex Offender Serves as a central repository that 2004 
Registry provides current information about 
convicted sex offenders. 
Canadian Firearms Program Provides operational support to the Transferred to 
prevention and investigation of RCMP 
firearms crime and misuse. in 2006 


Source: Royal Canadian Mounted Police 
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5.8 These services depend on the commitment of provinces and 
municipalities to input and share criminal information, as well as on 
federal financial support. 


§.9 | Partners—members of Canada’s law enforcement community 
that use or have input into national police services—have recognized 
the benefit of a central national repository and coordination of 
criminal information and investigational support activities. By 
providing these specialized training, technological, and scientific 
services, the RCMP enables uniform access to information that 
supports the administration of justice to all Canadians, particularly 
where the same level of services may not be within the resources of a 
given province or territory. 


§.10 In the 2010-11 fiscal year, the RCMP allocated $527 million to 
these specialized policing services, including 3,260 full-time personnel 
out of a total RCMP staff of 29,840. 


What we found in our previous audits 


§.11_ In our April 2000 chapter on services for Canada’s law 
enforcement community, we reported on several issues regarding 
governance, funding, and timeliness of national police services and 
recommended that the government reach agreements with all partners 
on required services, funding arrangements, structure, and 
accountability. 


§.12 In our May 2007 chapter on the management of forensic 
laboratory services, we found that the RCMP’s Forensic Laboratory 
Services was not meeting turnaround targets in the delivery of its 
services and that clients lacked the opportunity to influence 
operations. 


5.13 Finally, our previous audits found a backlog of fingerprints 
awaiting processing and identification. In our March 2004 chapter on 
national security, we recommended that the RCMP find and 
implement a solution to deal with its fingerprint backlog. In our follow- 
up chapter in March 2009, we found that progress on the development 
of the RCMP’s proposed solution, the Real Time Identification project, 
was satisfactory, but a backlog remained in updating individuals’ 
criminal record information. 


Focus of the audit 


5.14 This follow-up audit assessed the progress that the federal 
government and the Royal Canadian Mounted Police have made 
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against issues related to our recommendations from Chapter 7 of our 
April 2000 Report, Chapter 3 of our March 2004 Report, and 
Chapter 7 of our May 2007 Report. We also assessed progress against 
other audit observations on the governance and funding of national 
police services found in the preceding chapters, and on the status of 
the Real Time Identification project outlined in Chapter 1 of our 
March 2009 Status Report. 


5.15 We included Public Safety Canada in this audit because it is 
responsible for developing policing policy at the federal level. Public 
Safety Canada represents the federal government in negotiating police 
services agreements and provides advice and support to the Minister in 
matters regarding the RCMP 


§.16 More information on the audit objectives, scope, approach, and 
criteria can be found in About the Audit at the end of this chapter. 


Observations and Recommendations 


National police services are not covered by formal agreements 


5.17 The Royal Canadian Mounted Police (RCMP) is required by 
legislation to provide some policing support services, such as the 
Canadian Firearms Program, Forensic Laboratory Services, the 
National DNA Data Bank, and the National Sex Offender Registry. 
However, there are other national police services being provided by the 
RCMP that are not legislated but were requested by the federal 
government, because these services support essential policing activities 
for the RCMP’s members and for the law enforcement community in 
general. These include the Canadian Bomb Data Centre, the 
Canadian Criminal Real Time Identification Services, the Canadian 
Police College, the Canadian Police Information Centre, Criminal 
Intelligence Service Canada, and the Violent Crime Linkage Analysis 
System. These services are also important because central collection 
and exchange of critical criminal information and intelligence beyond 
existing jurisdictions enhances the security of all Canadians. Partners 
recognize that the RCMP is well positioned to provide these services 
given its federal, provincial, territorial, and municipal policing 
experience. 


5.18 The proposal to expand national police services was discussed at 
the 1966 Federal—Provincial Conference of Attorneys General on 
Organized Crime and confirmed and implemented through subsequent 
Treasury Board decisions, but it was not formalized through 
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Biology casework analysis agreements— 
Agreements whereby the federal government and 
the provinces agree to share the costs of biology 
casework analysis (DNA) arising from criminal 
investigations of designated offences in the 
provinces. 
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agreements or a governance framework between the federal and 
provincial governments. Since then, national police services have 
continued to expand in response to the increasing scope and influence 
of criminal activity across national and provincial boundaries as well as 
to rapidly changing advancements in technology that support police. 
Today, approximately 70 percent of national police services are being 
accessed by police forces other than the RCMP. Even though the 
RCMP uses approximately 30 percent of national police services, it is 
currently assuming most of the costs of these services. Further, there 
are still no agreements on what services will be provided. We found 
that the RCMP’s role, responsibility, and accountabilities have not 
been clearly defined for the following: 


* deciding which national police services should be provided to 
other police forces; 


* deciding how these services should be funded; and 


¢ determining how provincial, territorial, and municipal partners 
should be consulted. 


5.19 We also found that the absence of formal agreements for national 
police services creates uncertainties regarding the role of partners. In 
most instances, there is no obligation on any police force under 
provincial jurisdiction to provide or share criminal information. 


§.20 In our 2000 chapter, we recommended that the government reach 
agreements with all partners on national police services, funding 
arrangements, structure, and accountability, to which the RCMP 
concurred. We examined whether such agreements were in place. Since 
that time, the provinces and the federal government have signed the 
biology casework analysis agreements. However, we have assessed 
progress as unsatisfactory because the government has not reached 
agreements on any other national police services (Exhibit 5.2). 


5.21. The Royal Canadian Mounted Police Act provides the Minister of 
Public Safety with the authority to enter into arrangements with 
provinces and municipalities regarding the use of RCMP services. Public 
Safety Canada is responsible for leading efforts to develop policing policy 
and federal—-provincial policing agreements. Recently, the RCMP and 
Public Safety Canada have initiated discussions to address the lack of 
clarity of the roles and responsibilities of the RCMP and partners for 
national police services. Federal, provincial, and territorial assistant 
deputy ministers responsible for policing have met to discuss the issue. 


§.22 We recognize that any agreement on national police services will 
require effort and time on the part of Public Safety Canada, the RCMP 
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and provincial and territorial representatives. But such agreements 
could clarify the roles and responsibilities of all partners for the 
delivery of national police services to best serve the needs of the 
criminal justice system. 


Exhibit 5.2 Progress in addressing our recommendation on national police services agreements 


stakeholders on the national services needed, funding 
arrangements, structure and accountability. 


' | Unsatisfactory 
(Recommendation 7.123 of the 2000 April Report of the Auditor ; 


General of Canada, Chapter 7, Royal Canadian Mounted Police— 
Services for Canada’s Law Enforcement Community) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, and the 
time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, and 
the time that has elapsed since the recommendation was made. 


5.23 Recommendation. Public Safety Canada, in consultation with 
the Royal Canadian Mounted Police, should work with partners to 
identify what national police services are required; to examine issues 
related to governance, accountability, and funding of national police 
services; and to propose a way forward for implementation. 


Public Safety Canada’s and the RCMP’s response. Agreed. Public 
Safety and the RCMP recognize that, due to the complexity of this 
issue, a long-term, concerted effort will be required, starting with the 
study, “A Feasible and Sustainable Model for Forensic Services 
Delivery in Canada,” commissioned by Public Safety Canada. It is 
further recognized that, in order to sustain national police services in 
the future, agreement needs to be reached with partners on what 
services are required, and that issues related to governance, 
accountability, and funding need to be addressed. These issues are 
directly linked to the significant work required to address the other 
recommendations within this audit, including 


¢ identifying the services offered as part of national police services; 


* undertaking an analysis that includes cost to deliver the services; 
what has been funded; cost recovery efforts to identify gaps; the 
governance structure, which would include the ongoing study on 
alternative service delivery models and the RCMP’s strategic 
discussion paper, and consultation with the provinces and 
territories; 
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* developing options and outlining next steps for consideration by 
the government; and 


* negotiating funding arrangements with partners. 


Overlapping with this study would be the RCMP’s National Police 
Services Renewal and Sustainability Initiative that is intended to 
provide an overall view of the delivery of national police services. 


National police services governance lacks partners’ representation 


§.24 During the Federal—Provincial Conference of Attorneys General 
on Organized Crime in 1966, the provinces suggested that 
representatives from different jurisdictions facilitate the exchange of 
information among organizations to avoid national police services 

. becoming exclusively an RCMP operation. Representatives were 
chosen in 1969 for the Advisory Committee for the Canadian Police 
Information Centre to allow the larger law enforcement bodies to 
participate in the decisions made regarding the centre. 


§.25 In 2000, we noted that because Forensic Laboratory Services are 
used by other members of the Canadian law enforcement community who 
_rely on these services, we recommended the establishment of a national 
forensic advisory committee. In our 2007 chapter, we also recommended 
the establishment of a mechanism for consulting with partners so they 
have an opportunity to influence laboratory services, priorities, and 
service standards. The RCMP has implemented these recommendations. 


5.26 However, as noted in Exhibit 5.3, we found that several of the 
other national police services still have no advisory body. Further, some 
advisory bodies meet irregularly. Advisory bodies allow partners to 
influence decisions and participate in the development of the services’ 
policies. For example, when the Canadian Police Information Centre 
advisory board was first established in 1969, the federal government 
intended that through this participation, other partners from the law 
enforcement community would feel a direct responsibility to 
contribute, maintain, and use these services. 


§.27 Following our 2000 chapter, the RCMP established a National 
Police Services Advisory Council whose mission is to provide strategic- 
level advice to the RCMP for all national police services. It consists of 
executive-level members drawn from the public, private, and 
professional sectors with extensive management and federal 
government expertise. Each member is named by the Commissioner of 
the RCMP for a period of three years. Although it met irregularly in 
the past, it has now been meeting regularly, as intended. 
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Exhibit 5.3 Several national police services have no partner advisory body 


Canadian Criminal Real Time No committee 
Identification Services 


Forensic Laboratory Services Advisory committee 


Canadidn Police College Advisory board 


Canadian Police Information Centre Advisory committee 


Criminal Intelligence Service Canada National Executive Committee (also has 
decision-making authority) 


—}— 


No committee 


Canadian Bomb Data Centre 


Violent Crime Linkage Analysis System. No committee 


National DNA Data Bank National Advisory Committee 


Canadian Police Centre for Missing and No committee 


Exploited Children 


National Sex Offender Registry No committee 


+ 


Canadian Firearms Program Advisory board 


National police services (general) National Police Services Advisory Council 


5.28 We found, however, that the advisory council's terms of 
reference do not require partner representation. While there are 
benefits derived from the current composition of the council, adding 
representation from Canada’s law enforcement community would also 
promote a direct responsibility to contribute, maintain, and use these 
services, in line with the intent of the 1966 conference. Overall, we 
found progress to be satisfactory in addressing our recommendations 
on advisory committee and client consultation (Exhibit 5.4). However, 
in our opinion, the RCMP can improve partner representation on 
several of its national police services advisory bodies. 


5.29 Recommendation. The Royal Canadian Mounted Police should 
ensure partner representation in all national police services that best 
meets the needs of the users of these services. 


The RCMP’s response. Agreed. The RCMP agrees that partner 
representation is important in all national police services in order 

to best meet the needs of the users of these services. A strategic 
discussion paper is expected to be completed by the end of the 
2010-11 fiscal year, as part of the National Police Services Renewal 
and Sustainability Initiative. This initiative will examine the existing 
national police service governance structure to understand the current 
situation and articulate viable options for delivery of the services. 
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Service standard—The measurable level of 
service that clients can expect to receive under 
normal circumstances (for example, answer 
telephone calls within two minutes). 
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The intended outcome of this activity is to ensure better partner 
representation in the national police services governance structure, 
allowing national police services to best meet their users’ needs. 


Exhibit 5.4 Progress in addressing our recommendations on advisory committee and client 
consultation 


The RCMP should establish a national forensic advisory 
committee. 


(Recommendation 7.42 of the 2000 April Report of the Auditor 
General of Canada, Chapter 7, Royal Canadian Mounted Police— 
Services for Canada’s Law Enforcement Community) 


Satisfactory 


The RCMP should establish a mechanism for consulting with 
clients so that they have an opportunity to influence lab services, 
priorities, and service standards. 


(Recommendation 7.79 of the 2007 May Report of the Auditor 
General of Canada, Chapter 7, Management of Forensic Laboratory 
Services—Royal Canadian Mounted Police) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, and the 
time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, and 
the time that has elapsed since the recommendation was made. 


The RCMP has not determined the detailed transaction costs of each of its national 
police services 


§.30 In order to properly manage its resources, negotiate agreements 
on funding arrangements with partners, and explore options for 
different delivery models, the RCMP needs to be able to determine the 
detailed transactional costs of each of its national police services. 
Accurate costing information is an essential element in designing, 
providing, and managing services in an efficient manner. Determining 
these costs would also assist the RCMP in defining its requirements in 
order to meet future increases in service volumes. We examined 
whether the RCMP had determined the detailed transactional costs of 
each of its national police services. 


5.31 As early as 1997, as the RCMP was examining various options 
for the funding of national police services, it recognized that any 
discussions of cost-sharing or cost recovery would require details 
regarding service standards and service costs. In particular, accurate 
transaction costs are needed for any negotiations on cost recovery. 
Since then, the RCMP has led or participated in a number of studies 
aimed at determining the costing of national police services, including 
the contributions of partners to these services. 
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5.32. In one such instance, biology casework analysis agreements that 
came into force in 2001 stipulated that participating provinces would 
pay 55 percent of the average national RCMP cost per biology case for 
designated offences as determined by a costing analysis performed for 
the Solicitor General of Canada. However, we found that revised 
agreements signed in 2007 were not based on the cost per biology case. 
These agreements provided only a fixed amount to be paid by 
provinces receiving these services. Public Safety Canada told us that 
the existing funding arrangement for these agreements does not reflect 
the actual costs of biology case service requests today. 


5.33 We examined the cost per biology service request prepared by the 
RCMP but found that the costs were underestimated because the 
calculations did not include all of the work performed. Therefore, the 
RCMP cannot provide information on the actual costs it is incurring to 
provide this service to other police forces. 


5.34 In response to our 2007 chapter recommendation regarding a 
review of the Forensic Laboratory Services, the RCMP agreed that it 
should perform a cost-benefit analysis for these services. Although the 
RCMP performed a review of Forensic Laboratory Services in 2009, it 
did not include a cost-benefit analysis. However, the review 
recommended that each element of the RCMP’s biology services’ 
business be costed, starting at the gross unit level and then moving 
down to individual sites and processes. This had not yet been done at 
the time of our audit, resulting in an unsatisfactory progress rating 
(Exhibit 5.5). In determining progress as being unsatisfactory, we 
looked only at whether the RCMP performed a cost-benefit analysis of 
various services. 


Exhibit 5.5 Progress in addressing our recommendation on a review of Forensic Laboratory Services 


The RCMP should conduct a review of the Forensic Laboratory 
Services to examine internal efficiencies, perform a cost/benefit 
analysis of various services, and examine the need for additional 
resources. 


(Recommendation 7.50 of the 2007 May Report of the Auditor 
General of Canada, Chapter 7, Management of Forensic Laboratory 
Services—Royal Canadian Mounted Police) 


Unsatisfactory 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, and the 
time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, and 
the time that has elapsed since the recommendation was made. 
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5.35 At the time of this audit, we found that the RCMP could not 
provide detailed transactional costs of its national police services, 
which limits its ability to properly manage its resources. The National 
Police Services Renewal and Sustainability Initiative (Exhibit 5.6) 
currently under way is expected to provide information on these costs, 
but it is to be completed during the 2011-12 fiscal year. 


Exhibit 5.6 The National Police Services Renewal and Sustainability Initiative is not yet completed 


The National Police Services Renewal and Sustainability Initiative is a project that 
examines national police services. The project is designed to reaffirm the services’ 
identity and mandate, provide a complete financial and program overview, and provide 
a thorough analysis of potential strategies to optimize current systems and to develop 
new funding mechanisms for national police services in the future. Project results are 
to be prepared and submitted within the 2011-12 fiscal year. The project includes the 
following objectives: 


Review and validate the mandate of national police services. 


Review and validate the selection of services. 


Compare historical demands with current operational capacity and quantify the 
resource gap. 


Conduct a rigorous financial review of national police services. 


Review the governance and accountability structure for national police services. 


Develop and analyze potential mechanisms to provide dedicated and sustainable 
funding for national police services. 


Source: Adapted from the project charter of the National Police Services Renewal and Sustainability 
Initiative 


5.36 Recommendation. The Royal Canadian Mounted Police should 
complete as scheduled its National Police Services Renewal and 
Sustainability Project, including a detailed estimate of its transactional 
costs to provide each national police service, and monitor these costs 
Over time. 


The RCMP’s response. Agreed. The RCMP agrees that it should 
complete the National Police Services Renewal and Sustainability 
Initiative, which is currently continuing as planned. One of the 
intended outcomes of the National Police Services Renewal and 
Sustainability Initiative is to allow the RCMP to build a more robust 
and accurate method of costing of services. As well, a strategic 
national police services discussion paper, to be finished within the - 
2010-11 fiscal year, will include a review of national police services 
and their challenges, an articulation of potential options for 
governance and service delivery models, and recommendations for a 
long-term strategic vision. The discussion paper will be used as a 
vehicle to engage partner agencies in discussions concerning the future 
of national police services. 


Status Report of the Auditor General of Canada—2011 


NATIONAL POLICE SERVICES—ROYAL CANADIAN MOUNTED POLICE 


Resources for national police services have not kept pace with growing service 
requests 


5.37 Since the 1966 federal-provincial conference, national police 
services have expanded, the number of service requests has increased, 
and the services have also become reliant on the use of increasingly 
complex information systems requiring substantial investments. All of 
these factors have contributed to the growing cost of delivering national 
police services. We examined whether the RCMP was allocating 
necessary resources to meet the needs of all national police services. 


5.38 As early as 1992, the RCMP recognized that there was an 
increasing gap between its available resources and the cost of 
delivering national police services to Canada’s law enforcement 
community. As shown in Exhibit 5.7, policing support service 
expenses, which include national police services, exceeded allocations 
for the last five fiscal years, reaching a gap of $97.5 million for 


the 2008-09 fiscal year. 


5.39 To alleviate these financial pressures, we found that the RCMP 
has, over the past five years, been using the unspent portion of its 
budget from preceding fiscal years to ease financial gaps for national 
police services in subsequent years. In previous years, the main source 
of these funds has been from less than expected spending on new 
recruits. 


Exhibit 5.7 RCMP spending on policing support services, including national police services, has 
exceeded allocated funding for the last five fiscal years 


$ Millions 
740 
690 
640 
590 
on) fe Allocated funding 
490 a Actual spending 


440 


390 


2005-06 2006-07 2007-08 2008-09 2009-10 


Fiscal year 


Source: Main Estimates and Public Accounts of Canada 
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5.40 However, we found that the successful recruitment of police 
officers for contract and federal policing has reduced the carry-forward 
of unspent funds. This has reduced the RCMP’s flexibility to reallocate 
resources to resolve funding pressures such as those in national police 
services. To compensate, for the 2009-10 fiscal year, the RCMP 
reduced the initial budgets for all its programs by five percent, with the 
exception of funds earmarked for contract policing and 

the 2010 Vancouver Olympics, and reallocated those funds mostly to 
national police services. For the 2010-11 fiscal year, the RCMP 
reduced the initial budgets for its programs by 10.4 percent to cover 
funding gaps, including national police services. 


5.41 The internal reallocation increased available funding for 
national police services while other RCMP programs, primarily federal 
policing, had to cut back. In particular, the RCMP’s Federal and 
International Operations Directorate—which has responsibility for 
organized crime investigations, border integrity, drug enforcement, and 
money laundering—has had to reduce its budget by more than 

$47.7 million or 8.4 percent in the 2010-11 fiscal year. 


§.42 The RCMP finds that it is becoming increasingly difficult to 
continue funding the rising costs of providing national police services 
through internal reallocation. In order to balance its resources and 
without additional funding, officials have told us that the RCMP must 
either reduce these services or permanently redirect funding from 
other RCMP programs and activities to sustain national police 
services. 


5.43 We found that the RCMP is developing a comprehensive 
reallocation strategy to assess existing pressures and then identify 
opportunities to permanently reallocate resources to the highest 
priority pressures. Senior officials told us that this review will be 
followed by decisions that could include the termination of programs 
that are not a legal obligation of the RCMP. including some national 
police services. 


5.44 However, the development and successful implementation of 

this strategy, or any other approach to dealing with funding shortfalls, 
first requires a clear understanding of the roles and responsibilities of 
the RCMP and its partners in deciding which national police services 


should be provided and how they should be delivered. 


5.45 The RCMP’s past readiness to take on the delivery of additional 
national police services is a demonstration of its commitment to 
deliver services that are needed for policing. But, as funding pressures 
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indicate, there are limits to the RCMP’s capacity. In its 

December 2007 report, the Task Force on Governance and Cultural 
Change in the RCMP appointed by the federal government to advise 
on strengthening the governance of the RCMP recommended, “The 
RCMP should implement measures to ensure that no commitment to 
deliver a new service or modify an existing service is made without 
assurance that the necessary human resources and equipment can be 
deployed without compromising the ability of the Force to fulfill its 
existing responsibilities . . . ” 


5.46 Another example of a funding challenge is the National DNA 
Data Bank. The DNA Identification Act came into force in 2000, 
requiring the RCMP to provide services for the DNA Data Bank 
without additional permanent resources. With the coming into force of 
amendments to the Act, the number of service requests to the 
National DNA Data Bank has more than doubled, from 

around 19,800 in the 2007-08 fiscal year to approximately 40,800 in 
the 2008-09 fiscal year. The RCMP has made internal reallocations to 
make resources available for the National DNA Data Bank. 


5.47 In Budget 2010, the federal government announced that it 
would-explore options for different delivery models of the RCMP’s 
Forensic Laboratory Services, including privatization. It believes, 

“A new approach should improve the timeliness of processing samples, 
ensure sound financial administration and increase research and 
development in forensic science.” A study of these options is currently 
being planned by Public Safety Canada as lead in the federal 
government’s efforts to modernize the management, governance, and 
review of the RCMP. The preliminary results of this study are 
anticipated in late 2011. 


5.48 However, the study does not include all national police services, 
only the Forensic Laboratory Services. Considering the RCMP’s 
ongoing efforts to clarify its national police service roles and 
responsibilities and to solve current funding shortfalls, a study of all 
national police services to explore options for different delivery models 
may provide guidance to both Public Safety Canada and the RCMP in 
addressing these issues. 


5.49 Recommendation. In addition to the study of Forensic 
Laboratory Services currently being conducted by Public Safety 
Canada, the Royal Canadian Mounted Police, in partnership with 
Public Safety Canada, should examine all national police services to 
determine whether different delivery models could provide a better 
service in an efficient and cost-effective manner. 
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Delivering national police services 
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The RCMP’s and Public Safety Canada’s response. Agreed. The 
RCMP and Public Safety Canada agree that an examination of all 
national police services is appropriate to determine whether different 
delivery models may be more efficient and effective. The RCMP’s 
National Police Services Renewal and Sustainability Initiative, along 
with the results of the Forensic Laboratory Study, will support the 
actions identified in the response to Recommendation 5.23 and will 
provide the foundation for future deliberations and discussions 
surrounding possible delivery options. 


Timely response remains a challenge despite improvements 


§.50 In our April 2000 and May 2007 chapters, we found that the 
Forensic Laboratory Services did not deliver services in a timely 
manner. We reported that these delays had a negative impact on 
investigations and resulted in increased investigation costs. In our 
May 2007 follow-up audit, we found some improvements, such as the 
introduction of turnaround time targets. However, we also found that, 
for the most part, the turnaround targets were not met and that clients 
lacked the opportunity to influence how Forensic Laboratory Services 
were provided. 


5.51 Previous audits had also noted weaknesses with the timely 
processing of fingerprints and criminal record updates. In our 

April 2000 and March 2004 chapters, we recommended that the 
RCMP establish turnaround targets and implement a solution to deal 
with its fingerprint and criminal record update backlogs. The RCMP 
agreed with these recommendations. In later follow-up audits, we 
noted that while satisfactory progress had been made in implementing 
some of these recommendations, certain backlogs remained and were 
increasing. 


5.52 Since then, the RCMP has introduced a number of initiatives to 
improve the timeliness of service delivery for some of these services. 
This audit focused on four specific services: Forensic Laboratory 
Services, the National DNA Data Bank, Canadian Criminal Real 
Time Identification Services, and Criminal Intelligence Service 
Canada. We examined whether the RCMP is complying with agreed to 
service levels in the delivery of its national police services. 


5.53 Forensic Laboratory Services. The RCMP has made progress in 
addressing deficiencies in its Forensic Laboratory Services and in 
providing clients with an opportunity to influence lab services 

(Exhibit 5.4). In 2008, the RCMP developed the forensic investigation 
process to improve service through a more flexible and efficient intake 
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process in order to meet the needs of investigators in a timely manner. As 
part of planning for the new process, the RCMP undertook client 
consultations to determine users’ needs and has engaged users 
throughout the implementation to validate the new model. It is currently 
in the third step of a four-step implementation and has found that the 
new process has already decreased backlogs and processing times. 


5.54 Based on our testing, we found that while the average forensic 
lab processing time increased from 66.1 working days in 2007 to 

73.9 working days in 2008, the average processing time decreased to 
57.5 working days in 2009. For 2010, we found that the average 
processing time had decreased to 37.5 working days, which supports 
the RCMP’s assertion that processing times have decreased since using 
this new process. 


5.55 National DNA Data Bank. The RCMP was mandated by 
legislation to maintain a National DNA Data Bank. We found that the 
data bank did not have formal service standards; however, turnaround 
times are tracked in order to identify areas of delay. 


§.56 A DNA service request requires the services of both the National 
DNA Data Bank and the Canadian Criminal Real Time Identification 
Services as it is not permitted under the DNA Identification Act to 
identify a DNA sample without a corresponding criminal record. 
Exhibit 5.8 illustrates how the identification process works. 


5.57 We tested a sample of files where a DNA sample had been 
identified, and found that, on average, it takes 9.2 working days for the 
National DNA Data Bank to initially process a DNA sample. This is 
consistent with the turnaround times reported by the RCMP. The 
sample is then searched against existing profiles, which can take much 
longer in order to obtain a match. 


5.58 The search time is dependent on some factors that are beyond 
the RCMP’s control. For example, DNA from a crime scene cannot be 
matched unless there is a corresponding profile on record or criminal 
events happen that yield more information, such as a sample taken 
from a subsequent crime scene months after an initial crime generates 
a match. RCMP officials indicated that the waiting time for this match 
could be years, or that a match may never be identified. We found that 
the time to complete the portion of the work undertaken by the 
National DNA Data Bank is consistent with the turnaround times 
reported by the RCMP. Once a match is found, it then takes Canadian 
Criminal Real Time Identification Services, on average, 12.3 days to 
complete the work and return the results to the client. 
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Exhibit 5.8 The DNA sample identification process involves the National DNA Data Bank and the 
Canadian Criminal Real Time Identification Services 


Forensic laboratories process s2 
inspections 
profiles into the Crime Scene Index (CSI) 

of the National DNA Data Bank. 


Source: Adapted from the National DNA Data Bank Annual Report 


§.59 Canadian Criminal Real Time Identification Services. We 
continue to find delays with updates to criminal records. These records 
are maintained by the Canadian Criminal Real Time Identification 
Services. In our March 2009 chapter, we reported that although the 
backlog for checking fingerprints against the database of existing 
fingerprints had been eliminated, a backlog remained in updating 
individuals’ criminal record information. This backlog has worsened. 
Since the 2005—06 fiscal year, the outstanding criminal record update 
backlog has increased from 186,018 pages (each criminal record 
contains one or more pages of information) to more than 1.4 million 
in 2010 (Exhibit 5.9). The estimated time to process a criminal record 
update is 14 months for English updates and 36 months for French 
updates. 
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Exhibit 5.9 Criminal record update backlog has more than quadrupled since the 2005-06 fiscal year 


Number of outstanding pages (thousands) 
1,600 
1,400 
1,200 
1,000 
800 
600 


400 


2005-06 2006-07 2007-08 2008-09 2009-10 October 2010 


Fiscal year 


Source: Royal Canadian Mounted Police 


5.60 We undertook a review of a representative sample of files to 
determine how long it takes to process a new criminal record and to 
update an existing criminal record. For a new record, the average time 
to process was 27 working days. This is significantly higher than the 
optimal response time of two hours that the RCMP expects to achieve 
upon completion of the Real Time Identification project. For updates 
to existing criminal records, we found that only 30 percent of the files 
originally selected as part of our sample had been completed, with the 
remaining 70 percent in different stages of the process. For the files 


that had been completed, we found that the average time to update an 


existing criminal record was 334 working days. RCMP officials 
explained that this delay results from significantly increased demand. 
The RCMP expects to achieve a response time of 24 hours for an 
existing criminal record update upon completion of the Real Time 
Identification project. 


5.61 As reported in our March 2009 chapter, the Real Time 
Identification project was supposed to help reduce the criminal record 
update backlog. At that time, the RCMP indicated that the project was 
progressing well and was on budget. However, the project is no longer 
on schedule and was on hold at the time of our audit pending a review 
of its scope. Therefore, progress toward the implementation of the Real 
Time Identification project is unsatisfactory (Exhibit 5.10). Accurate, 
up-to-date criminal records and fingerprints are crucial to Canada’s 
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law enforcement. In previous reports, we noted that Canada’s 
fingerprint and criminal history records system were unable to keep up 
with demand and recommended that the RCMP implement a solution. 
The Real Time Identification project was proposed to improve the 
efficiency of Canada’s national fingerprint repository (phase 1) and 
criminal record repository (phase 2). 


Exhibit 5.10 Progress in addressing our recommendations on the Real Time Identification project 


The RCMP and Public Safety and Emergency Preparedness 
Canada should give priority to implementing the Real Time 
Identification project. 


(Recommendation 3.109 of the 2004 March Report of the Auditor 


General of Canada, Chapter 3, National Security in Canada—The 
2001 Anti-Terrorism Initiative) 


Unsatisfactory 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, and the 
time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, and 
the time that has elapsed since the recommendation was made. 


5.62 In 2005, the RCMP received additional funding in the amount of 
$99.7 million and allocated a further $30 million of its existing 
resources to fund the project, for a total of $129.7 million. Delays and 
complexity impacted the timely delivery of phase 1. As reported in our 
March 2009 Status Report, phase | was concluded in September 2008, 
21 months later than planned. 


5.63 Phase 2 was scheduled to begin in the fall of 2006, but did not 
start until February 2008. However, issues from phase | remained 
unresolved and affected the progress of phase 2. An independent 
review of the project was undertaken in January 2010 due to the 
ongoing difficulties in deciding on the project scope, as well as cost 
overruns and schedule delays. The results of the review led to a 
decision by the RCMP to issue a stop work order for phase 2 on 

25 May 2010, until further notice, to reassess the project scope, 
timeline, and funding. At the time of our audit, the project was still on 
hold, and the RCMP was in the process of revising the project scope. 


5.64 Criminal Intelligence Service Canada. Part of Criminal 
Intelligence Service Canada’s mandate is to ensure timely production 
and exchange of criminal information and intelligence products. The 
Automated Criminal Intelligence Information System (ACIIS), 
managed by Criminal Intelligence Service Canada, is the Canadian law 
enforcement community’s national database for criminal information 
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and intelligence on organized and serious crime. It facilitates the 
sharing of criminal intelligence between law enforcement agencies. 
In 2008, Criminal Intelligence Service Canada undertook a review of 
ACIIS, including consultation with law enforcement personnel and 
intelligence analysts from all levels across Canada. It was determined 
that deficiencies in the existing ACIIS were eroding trust in the system 
and creating doubts as to its value. The review also found that the 
system is cumbersome, requiring manual data re-entry of information 
already contained within police forces’ own local systems. Where law 
enforcement resources are stretched, there is insufficient time and 
resources to duplicate efforts by entering the same information into 
two systems. 


§.65 ACHS also offers limited analysis capability. For Criminal 
Intelligence Service Canada, sharing information is essential in its 
strategy to fight organized crime. The total number of ACIIS 
information transactions has doubled between 2006 and 2008. 
Criminal Intelligence Service Canada told us that the current ACIIS 
system will soon no longer be supportable. Criminal Intelligence 
Service Canada estimates that an upgrade to a new criminal 
intelligence system will cost more than $70 million. No source of 
funding has been identified to meet this need. 


§.66 Despite improvements made in some areas of national police 
services, timely responses to client requests remain a challenge and the 
backlog for updating criminal records has increased since our 

March 2009 Status Report. Demand has continued to increase 
(Exhibit 5.11) and proposed changes to the DNA Identification Act 
have the potential to further increase the number of service requests to 
be processed each year. Combined with existing financial pressures, this 
increase in backlogs and service requests is straining the RCMP’s ability 
to continue delivering national police services in a timely manner. 


Exhibit 5.11 Client service requests continue to increase 


Time Identification 


Canadian Criminal Real Criminal requests 753,687 880,475 967,440 916,267 972,865 930,790 


+. 


Services Civil requests | 194,782 | 199,161 | 212,240 | 245,055 | 242,088 | 241,894 


National DNA Data Bank 


17,429 19,048 19,613 19,822 40,733 | 42,879 


Forensic Laboratory Services 


11,387 11,746 le A isey 11,291 a2 12,618 
eb eee 


Source: Compiled by the Office of the Auditor General using data provided by the RCMP 
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5.67 In addition to the absence of any legal requirement for police 
forces to provide or share most criminal information, these increases in 
backlogs and processing delays limit the RCMP’s ability to provide 
assurance that its various national police services’ repositories, such as 
fingerprints, criminal records, and organized crime intelligence, 

are complete and up to date. Much work remains to be done to resolve 
these backlogs and delays. 


Conclusion 


5.68 National police services provided by the RCMP are valuable to 
the Canadian criminal justice system. They support the national 
coordination of activities and information through the delivery of 
specialized training as well as technological and scientific services that 
ensure a uniform level of public safety and the administration of justice 
to all Canadians, particularly where such services may not be within 
the resources of provincial and territorial jurisdictions. 


5.69 The RCMP has made satisfactory progress in providing its 
partners and clients with opportunities to influence national police 
services, priorities, and service standards. However, specific services 
still do not provide such an opportunity, while the National Police 
Services Advisory Council does not include partner representation. 


§.70 Despite a commitment to do so, the RCMP has not made 
satisfactory progress in implementing recommendations regarding the 
timely delivery of national police services from our 2000, 2004, 

and 2007 reports. While the RCMP has improved the timely delivery 
of some of its national police services, it still faces increased backlogs 
and delays in the update of criminal records. Technological upgrades to 
critical national police services systems such as the Real Time 
Identification project either face significant challenges or are being 
delayed. This increases the risks of loss or delays in the timely 
transmittal of critical criminal information that could affect the safety 
and security of Canadians. 


5.71 The federal government has not made satisfactory progress in 
implementing our 2000 recommendation to reach agreements with 
partners on national police services requirements, funding 
arrangements, and accountability. The absence of formal agreements 
creates uncertainties regarding national police services roles and 
responsibilities of the RCMP and its partners in deciding which 
national police services should be provided; how these services 
should be funded; and how provincial, territorial, and municipal 
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partners should be engaged within the national police services 
governance framework. 


5.72 We found that the RCMP has been challenged to continue 
paying for the rising costs of national police services. The increasing 
scarcity of unspent funds in other areas to cover financial shortfalls in 
national police services could affect the RCMP’s ability to deliver these 
services along with its other obligations. The RCMP has begun to take 
action by assessing existing financial pressures and then examining and 
identifying opportunities for permanent reallocation of resources. 
However, the development and successful implementation of this or 
any other approach in dealing with funding shortfalls will first require a 
clear understanding of the roles and responsibilities of the RCMP and 
its partners in deciding which national police services should be 
provided, what their costs are, and how they should be funded. 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these 
standards as the minimum requirernent for our audits, we also draw upon the standards and practices of 
other disciplines. 


Objectives 


The objective of our audit was to determine whether the federal government and the Royal Canadian 
Mounted Police (RCMP) have made satisfactory progress in implementing selected recommendations 
from Chapter 7 of our 2000 April Report, Chapter 3 of our 2004 March Report, and Chapter 7 of our 
2007 May Report. 


Scope and approach 


Our audit covered the planning, management, and delivery of the Royal Canadian Mounted Police’s 
national police services as provided by Policing Support Services. The audit scope for our examination of 
the criteria regarding compliance to service levels focused exclusively on the following national police 
services: 


« Forensic Laboratory Services 
+ National DNA Data Bank 
* Canadian Criminal Real Time Identification Services 
¢ Criminal Intelligence Service Canada 
The remainder of the audit examined all national police services. 


The audit team interviewed staff from the RCMP mostly senior management and analysts from Policing 
Support Services and the various national police services. Other personnel interviewed included 
representatives from the Chief Information Officer and Chief Financial Officer branches. 


A statistically representative sample of 57 records was randomly extracted from the population of 

97,492 completed National DNA Data Bank records in the scope of our audit, with a date range from 
April 2007 to September 2010. Only records that matched Canadian Criminal Real Time Identification 
Services entries during the same date range were included as part of the audit sample. This sample is 
sufficient to conclude on the sampled population with a confidence level of 90 percent, and a confidence 
interval of +10 percent, 19 times out of 20. 


New and updated records were sampled separately from the fingerprint and criminal record system, with a 
date range from April 2007 to September 2010. New records are those that did not match any existing 
fingerprint or criminal record, while updated records are those for which matches have been. 

found. Separate statistically representative samples of 59 updated records and 60 new records were 
randomly extracted from the population within the scope of our audit of 227,019 updated and new 
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records. Both samples are sufficient to conclude on the sampled populations with a confidence level of 
90 percent, and a confidence interval of +10 percent, 19 times out of 20. 


Sampling was not performed on the Laboratory Information Management System (LIMS) database for 
Forensic Laboratory Services. Rather, all measures were calculated on the entire population in the scope of 
our audit, which consisted of 103,658 service requests, filtering out those that were either case 


management, case receipt, or queued entries. The date range for the sample was from April 2007 to 
September 2010. 


) 


Specific audit questions were also directed to Public Safety Canada for its role in developing appropriate 
national policies for new and evolving crime and border issues and in contract negotiations with provinces 
and territories for national police services, and the Department of Justice Canada for its role in the 
preparation of proposed amendments to the DNA Identification Act, which may have a bearing on costs 
and service levels. 


The audit did not examine the performance of provinces and territories. 


Criteria 


The RCMP has legislation in place that gives it authority and * Royal Canadian Mounted Police Act, section 18 
responsibility for the delivery of national police services. 


DNA Identification Act, section 5 


Firearms Act, section 81.2 


Sex Offender Information Registration Act, sections 13 
and 14 


. 


Order Transferring from the Canadian Firearms Centre to the 
Royal Canadian Mounted Police the Control and Supervision 
of the Canadian Firearms Centre 


Agreements between the federal government and provincial 
and territorial governments affecting national police services 


Policing services agreements between the federal government 
and provincial and territorial governments (contract policing) 


Treasury Board decisions 


National police services funding is aligned with legislation and ¢ Various appropriation acts 


SUP POrINE auinag mes ocd valtangawents: * Treasury Board decisions pertaining to national police services 


Agreements between the federal government and provincial 
and territorial governments affecting national police services 


Policing services agreements between the federal government 
and provincial and territorial governments (contract policing) 


Common Services Policy, sections 5.6 and 5.7, Treasury 
Board, 2006 
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¢ Service Standards: A Guide to the Initiative, Treasury Board, 
1995 


* Quality and Affordable Service for Canadians: Establishing 
Service Standards in the Federal Government—An Overview, 
Treasury Board, 1994 


The RCMP complies with legislated, regulated, and agreed-to 
service levels in the delivery of national police services. 


Management reviewed and accepted the suitability of the criteria used in the audit. 


Period covered by the audit 


With the exception of the criteria regarding authority to deliver, for which the audit period started with 
events leading to the January 1966 Federal—Provincial Conference of Attorneys General on Organized 
Crime, the period audited for this chapter is from the beginning of the fiscal year 2006-07 

until 1 November 2010. 


Audit work for this chapter was substantially completed on 1 November 2010. 


Audit team 


Assistant Auditor General: Wendy Loschiuk 
Principal: Gordon Stock 
Director: Jean Goulet 


Maryanna Basic 
John Hilton 
Catherine Martin 
Nicolette O’Connor 
Stacey Wowchuk 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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Appendix List of recommendations 


The following is a list of recommendations found in Chapter 5. The number in front of the 
recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses 
indicate the paragraphs where the topic is discussed. 


Governing national police services 


§.23 Public Safety Canada, in Public Safety Canada’s and the RCMP’s response. Agreed. 
consultation with the Royal Canadian Public Safety and the RCMP recognize that, due to the 
Mounted Police, should work with complexity of this issue, a long-term, concerted effort will be 


partners to identify what national police —_ required, starting with the study, “A Feasible and Sustainable 
services are required; to examine issues | Model for Forensic Services Delivery in Canada,” commissioned 


related to governance, accountability, by Public Safety Canada. It is further recognized that, in order to 
and funding of national police services; sustain national police services in the future, agreement needs to 
and to propose a way forward for be reached with partners on what services are required, and that 
implementation. (5.17—5.22) issues related to governance, accountability, and funding need to 


be addressed. These issues are directly linked to the significant 
work required to address the other recommendations within this 
audit, including 


* identifying the services offered as part of national police 
Services; 


undertaking an analysis that includes cost to deliver the 
services; what has been funded; cost recovery efforts to 
identify gaps; the governance structure, which would 
include the ongoing study on alternative service delivery 
models and the RCMP’s strategic discussion paper, and 
consultation with the provinces and territories; 


developing options and outlining next steps for 
; consideration by the government; and 


* negotiating funding arrangements with partners. 


Overlapping with this study would be the RCMP’s National 
Police Services Renewal and Sustainability Initiative that is 
intended to provide an overall view of the delivery of national 
police services. 
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5.29 The Royal Canadian Mounted 
Police should ensure partner 
representation in all national police 


services that best meets the needs of the 
users of these services. (5.24—5.28) 


The RCMP’s response. Agreed. The RCMP agrees that partner 
representation is important in all national police services in 
order to best meet the needs of the users of these services. A 
strategic discussion paper is expected to be completed by the end 
of the 2010-11 fiscal year, as part of the National Police Services 
Renewal and Sustainability Initiative. This initiative will 
examine the existing national police service governance 
structure to understand the current situation and articulate 
viable options for delivery of the services. 


The intended outcome of this activity is to ensure better partner 
representation in the national police services governance 
structure, allowing national police services to best meet their 
users’ needs. 


Funding national police services 


5.36 The Royal Canadian Mounted 
Police should complete as scheduled its 
National Police Services Renewal and 
Sustainability Project, including a 
detailed estimate of its transactional 
costs to provide each national police 
service, and monitor these costs over 


time. (5.30-5.35) 


5.49 In addition to the study of 
Forensic Laboratory Services currently 
being conducted by Public Safety 
Canada, the Royal Canadian Mounted 
Police, in partnership with Public Safety 
Canada, should examine all national 
police services to determine whether 
different delivery models could provide 
a better service in an efficient and cost- 
effective manner. (5.37-5.48) 


The RCMP’s response. Agreed. The RCMP agrees that it 
should complete the National Police Services Renewal and 
Sustainability Initiative, which is currently continuing as 
planned. One of the intended outcomes of the National Police 
Services Renewal and Sustainability Initiative is to allow the 
RCMP to build a more robust and accurate method of costing of 
services. As well, a strategic national police services discussion 
paper, to be finished within the 2010-11 fiscal year, will include a 
review of national police services and their challenges, an 
articulation of potential options for governance and service 
delivery models, and recommendations for a long-term strategic 
vision. The discussion paper will be used as a vehicle to engage 
partner agencies in discussions concerning the future of national 
police services. 


The RCMP’s and Public Safety Canada’s response. Agreed. 
The RCMP and Public Safety Canada agree that an examination 
of all national police services is appropriate to determine 
whether different delivery models may be more efficient and 
effective. The RCMP’s National Police Services Renewal and 
Sustainability Initiative, along with the results of the Forensic 
Laboratory Study, will support the actions identified in the 
response to Recommendation 5.23 and will provide the 
foundation for future deliberations and discussions sa aae 
possible delivery options. 
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Funding for the program has increased over time 

Health Canada is not meeting its obligations for the timely review of medical devices applications 
Health Canada does not know if it is meeting its obligations for its post-market activities 

Steps have been taken to address remaining funding shortfalls 


Approving medical devices for sale in Canada 

Health Canada’s decisions on medical device applications are not timely 

Untimely decisions can have important consequences 

Health Canada could benefit from increased international cooperation for its pre-market activities 
Managing risks from devices available in Canada 


Health Canada has significantly expanded its post-market activities 


Health Canada identifies risks for key post-market activities but does not know if they are 
adequately mitigated 

Health Canada is working cooperatively with international jurisdictions on post-market activities 
but needs to determine if further actions are necessary 
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What we examined 


Why it’s important 


Regulating Medical Devices— 
Health Canada 


Main Points 


The term “medical devices” encompasses a wide range of medical, 
surgical, and dental products and instruments used to diagnose, treat, 
and prevent diseases and other physical ailments. They range from 
basic items, such as bandages, to complex devices such as pacemakers 
and diagnostic ultrasound systems. 


As required under the Food and Drugs Act, Health Canada regulates 
the safety and effectiveness of all medical devices marketed in Canada. 
It does this through a combination of scientific review, monitoring, 
compliance, and enforcement activities, both before and after the 
devices reach the Canadian marketplace. The Department works 
toward ensuring that the public has timely access to safe and effective 
medical devices, and that those who need to know about related safety 
concerns are informed of them in a timely manner. 


In March 2004 we reported concerns about the continued viability of 
the Medical Devices Program, given the increasing complexity and 
number of devices available in Canada and the resources allocated to 
the Program. We concluded that the Program was not sustainable with 
the resources it had. We made similar observations in our 


November 2006 Report. 


In this audit, we examined Health Canada’s progress in meeting 
commitments it made in response to selected recommendations from 
our previous reports. In particular, we examined the timeliness of the 
Department’s review of applications to allow or deny medical devices 
to be sold in Canada. We also examined how it manages the risks 
related to medical devices already available on the Canadian market 
and how it meets its obligations for medical devices. In addition, we 
looked at international regulatory cooperation. 


Audit work for this chapter was substantially completed on 
29 October 2010. 


Medical devices play an important role in all stages of the delivery 
of quality health care, from the most basic procedures, such as 
monitoring blood pressure and body temperature, to complex surgeries. 
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What we found 


All are meant to improve the health and well-being of patients by 
helping to prevent, diagnose, and treat disease, to reduce pain and 
suffering, and to extend and save lives. Medical devices are increasing 
in number and complexity, due in part to medical and technological 


advances. 


Close to 1.4 million different medical devices are currently on the 
Canadian market. Canadians rely on Health Canada for assurance that 


available medical devices are safe and effective. Users of medical 
devices also rely on Health Canada to promptly inform them of safety 
concerns related to medical devices that are in use. 


Overall, Health Canada has made unsatisfactory progress toward 
meeting commitments it made in response to recommendations in 
our 2004 and 2006 reports. Although it has accomplished much in 
some areas, the Department is not meeting its obligations under the 
Medical Devices Program, mostly at the pre-market stage. For 
example, more than 45 percent of the time it does not meet its 
service standards for timely review of medical device submissions, 
thus delaying Canadians’ access to the health benefits of these 
devices. According to Health Canada, this problem is due to a 
funding shortfall caused in part by rapid growth in medical device 
technology and increasingly complex submissions. 


Health Canada does not know if it has allocated too much or not 
enough resources to post-market activities such as inspections and 
surveillance. The Department has not established what levels of 
activity are needed to protect the health and safety of Canadians. In 
addition, while the Department has identified risks associated with 
medical devices already available on the Canadian market, it has yet 
to determine whether the risks that the inspections and the review of 
incident reports are designed to address have been adequately 
mitigated. 


Health Canada recently developed strategies to increase cooperation 
at the pre-market stage with regulatory bodies in other countries. 
While we acknowledge the significant challenges involved, progress 
to date is unsatisfactory. The work has not taken advantage of 
foreign resources, information, and knowledge and has not yet 
improved regulatory performance. 


Health Canada has taken significant actions to meet some 
commitments made in response to other recommendations from 
previous reports. A major accomplishment was making the Medical 
Devices Program more sustainable than in previous years. The 
Department increased its funding to the Program and expects to 
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eliminate the remaining funding shortfall by implementing new 

user fees starting in the 2011-12 fiscal year. Another significant 
accomplishment has been establishing a national inspection program 
to assess compliance with the Food and Drugs Act and Regulations. 
The Department also increased its capacity to identify safety risks 
associated with medical devices on the market. 


The Department has responded. The Department agrees with all of 
the recommendations. Its detailed responses follow the 
recommendations throughout the chapter. 
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Introduction 


Medical devices in Canada 


6.1 Canadians rely on a range of medical devices to prevent, 
diagnose, and treat diseases, disabilities, and other physical conditions. 
Devices such as dialysis equipment, pacemakers, and surgical lasers 
have improved or saved the lives of many Canadians and are 
indispensable to today’s health care practitioners. 


6.2 In Canada, the safety and effectiveness of medical devices are 
regulated under the Food and Drugs Act administered by Health 
Canada. The Act defines a medical device as “any article, instrument, 
apparatus or contrivance, including any component, part or accessory 
thereof, manufactured, sold or represented for use” in the medical 
treatment of human beings. 


6.3. Medical devices are categorized in four classes (I, II, II, and IV) 
on the basis of the risks associated with their use and the controls 
necessary to provide reasonable assurance that they are safe and 
effective. Exhibit 6.1 lists the various classes of medical devices and 
the risk levels associated with their use. 


Exhibit 6.1 Risk levels associated with classes of medical devices 


Reusable surgical scalpel, bandages, 
culture media 


Class | Lowest 


Contact lenses, epidural catheters, 


ony Bob pregnancy test kits, surgical gloves 


Orthopaedic implants, glucose monitors, 
Class III Moderate dental implants, haemodialysis systems, 
diagnostic ultrasound systems 


HIV test kits, pacemakers, angioplasty 


Class IV High catheters 


Source: Health Canada 


Stakeholders and partners in medical devices 


64 In addition to the federal government, in its role as regulator, 
there are a number of other important stakeholders and partners in 
medical devices. 


6.5 The public. The primary stakeholder is the public. Members of 
the public buy and use medical devices to receive some health benefit. 
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However, with this benefit comes some level of potential risk, since the 
quality and safety of devices can never be absolutely guaranteed. The 
Canadian public has limited control over the risks and benefits to 
which it is exposed. Ultimately, Canadians must trust others, such as 
health care professionals and manufacturers of therapeutic products, to 
provide safe and effective services and products. 


6.6 Health care facilities and health care professionals. Health 
care facilities, which are under the responsibility of provinces and 
territories, buy and use medical devices. Health care professionals 
provide advice on how to use or install medical devices. They also play 
a significant role in helping to ensure the continued safety of medical 
devices for the public. They are often the first to become aware of a 
problem with a device; this makes them a critical source of information 
on adverse incidents. They can help prevent future problems from 
occurring by passing this information on to manufacturers and/or the 
federal government in a timely manner. 


6.7. Medical devices industry. The legal responsibilities of the 
medical devices industry are outlined in the Medical Devices 
Regulations, which place significant responsibility on manufacturers, 
importers, and distributors to do all they can to protect the health and 
safety of the public. 


6.8 Foreign regulatory authorities. Health and safety concerns are 
shared by governments worldwide. Health Canada works with foreign 
regulators to develop international standards, harmonize regulations, 
and share information on adverse incidents involving medical devices. 
Participation in such activities can help the Department increase its 
capacity to make timely regulatory decisions about medical devices 
before they are sold in Canada, and to quickly identify risks associated 
with devices already on the Canadian market. 


Health Canada’s Medical Devices Program 


6.9 Health Canada’s Medical Devices Program is responsible for 
regulating the safety and effectiveness of medical devices both before 
and after they reach the Canadian marketplace. The program uses a 
regulatory framework based on risk management principles and 
scientific methods and seeks to emphasize public safety, while 
optimizing quality and effectiveness. The Department’s responsibilities 
apply to all medical devices marketed in Canada, regardless of whether 
they are manufactured here or overseas. 
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6.10 The program uses a combination of scientific review, monitoring, 
and compliance and enforcement activities to pursue its goal of ensuring 
that medical devices available in Canada are safe and effective. 


6.11 The Medical Devices Program is delivered jointly by the Health 
Products and Food Branch (composed of the Therapeutic Products 
Directorate, the Health Products and Food Branch Inspectorate, and 
the Marketed Health Products Directorate) and the Regions and 
Programs Branch. In the 2009-10 fiscal year, direct costs of the 
program amounted to $20.9 million. Its staff consisted of 185 full-time 
equivalent employees. 


6.12 Pre-market activities. The Medical Devices Program regulates 
medical devices and their manufacturers, importers, and distributors, 
through two distinct licensing regimes: licensing of establishments and 
licensing of medical devices. 


6.13 Health Canada requires manufacturers of Class I devices that do 
not sell through an establishment already holding a licence, as well as 
importers and distributors of any medical devices for human use, to 
obtain an establishment licence to sell their products in Canada. This 
requirement applies to organizations located in Canada and abroad. 
Establishment licences attest that the organizations holding them 
comply with regulatory requirements, and that those organizations 
have documented procedures to manage complaints, recalls, shipping, 
storage, and delivery of medical devices marketed in Canada. Health 
Canada issued 2,048 establishment licences in the 2008-09 fiscal year 
and 2,147 in the 2009-10 fiscal year. Establishment licences have to be 
renewed once a year. Because most of the establishment licences issued 
by Health Canada were renewals as opposed to new ones, both we and 
the Department considered this activity to be a post-market activity. 


6.14 The Medical Devices Regulations require that manufacturers who 
are seeking a licence for a Class II, II, or IV device first demonstrate to 
Health Canada that they meet quality standards in the design and 
manufacturing of their medical devices. These standards are based on 
internationally accepted standards for quality systems. 


6.15 Before issuing a device licence, Health Canada evaluates 
information provided by manufacturers in support of their claim that 
the device meets the safety and effectiveness requirements of the 
Medical Devices Regulations. The nature of the information required 
from manufacturers and evaluated by Health Canada depends on the 
risk rating of the device. If a high-risk device is modified, Health 
Canada must also authorize the change. As a result, Health Canada 
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reviews license applications for both new and modified devices. The 
average number of Class II, III], and IV medical device applications 
reviewed annually by Health Canada over a five-year period (fiscal 
years 2005—06 to 2009-10) was 4,007. 


6.16 Post-market activities. The pre-market review process cannot 
identify or predict all possible malfunctions of the more than 

1.4 million different medical devices currently on the Canadian 
market. Consequently, both industry and Health Canada must 
continually monitor and assess the safety of medical devices on the 
market, to protect the health and safety of Canadians. To do so, Health 
Canada receives reports of suspected problems from manufacturers, 
health care professionals, and consumers. The Department assesses the 
severity of the reported problems and takes action to mitigate the 
health and safety risks when necessary. 


6.17. The Department also monitors recalls of medical devices by 
domestic and foreign manufacturers and conducts inspections of 
establishments that manufacture, import, or distribute medical 
devices. It monitors various sources of scientific and medical 
information to identify problems and potential risks related to medical 
devices on the market. Health Canada evaluates information that 
stems from these activities and takes action if it identifies a serious 
health risk. Such actions can include 

* requiring that a manufacturer modify product labels and 

instructions on how to use devices, 


* issuing warnings to the public and the health care community 
about potential device-related health and safety concerns, and 


* advising industry to remove a product from the market. 


What we found in our 2004 and 2006 audits 


6.18 In March 2004, we concluded that the program was not 
sustainable in its (then) current state. We recommended that Health 
Canada review applications for medical devices, on a timely basis, and 
find ways to increase international regulatory cooperation. At the post- 
market stage, we recommended that Health Canada engage in active 
inspection of industry operations to verify compliance with regulations. 
We also recommended that it improve its post-market surveillance 
system to provide timely, accurate, and complete information about 
incidents involving medical devices. 
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6.19 In November 2006, as part of a broader examination of the 
allocation of resources to regulatory programs managed by Health 
Canada, we expressed concerns about the Department’s ability to meet 
its obligations for medical devices. We found that the Department had 
not determined the level of activity required to meet its 
responsibilities. The Department committed to review the funding 
allocated to the Medical Devices Program and to renew its cost- 
recovery regime. It also committed to establish the level of activity 
required to meet its responsibilities for medical devices—along with 
corresponding performance targets and resource requirements. 


Focus of the audit 


6.20 The focus of our audit was to determine whether Health Canada 
has made satisfactory progress in implementing selected 
recommendations we have made on its management of the Medical 
Devices Program. 


6.21 Because our 2004 audit concluded that the pre-market 
evaluation process was consistently followed, we focussed our audit on 
the timeliness of this process. We also examined the management of 
risks from medical devices already on the Canadian market. We 
focused our audit on four post-market activities: 


* inspections, 

* the review of incident reports, 

* signal assessment and surveillance, and 
* risk communication. 


6.22 We also looked at actions taken by the Department to fund the 
program and to determine the level of activity required to meet its 
obligations for medical devices. Finally, we examined actions taken by 
the Department to increase international cooperation. We did not 
examine the monitoring of investigational testing (clinical trials), 
unlicensed devices, and the reuse of single-use devices. Also, we did 
not audit the process for allocating resources reported on in our 


2006 report. 


6.23 More details on the audit objectives, scope, approach, and 
criteria are in About the Audit at the end of this chapter. 
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Direct costs—Costs of the Medical Devices 
Program that are directly related to the service 
activity, for example, the cost of an evaluator to 
review an application. 
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Observations and Recommendations 


Funding for the program has increased over time 


6.24 The sustainability of a program depends on both the amount and 
sources of its funding. Health Canada funds its ongoing program 
activities, such as regulatory compliance and enforcement, through 
public monies authorized by Parliament. Because the activities of 
Health Canada’s Medical Devices Program also benefit the private 
sector, the government introduced user fees in 1996. Revenue from 
industry is used to pay for the costs of running the program and to 
provide services at specified levels. 


6.25 Our 2004 audit noted that insufficient resources were allocated 
to deliver the Program. For the 2002-03 fiscal year, program funding 
for the direct costs of pre-market activities was $1.7 million plus 

$1 million for post-market activities, for a total of $2.7 million. The 
Department's funding was supplemented by user fees charged to 
industry to recover some costs. In the same fiscal year, Health Canada 
recovered $4.6 million. This brought its total program funding for 
direct costs to $7.3 million. 


6.26 Our 2006 audit showed that program funding for direct costs had 
increased to $8.8 million in the 2003—04 fiscal year, to $11.4 million in 
the 2004-05 fiscal year, and to $13.7 million in the 2005—06 fiscal year. 
We recommended that Health Canada review the funding allocated to 
regulatory programs to ensure programs received sufficient funding to 
meet the Department's obligations and to adequately protect the 
health and safety of Canadians. The Department agreed with our 
recommendation and committed to complete a review of funding 
allocated to regulatory programs by March 2008. 


6.27 We examined the funding allocated to the Medical Devices 
Program for the 2008-09 and 2009-10 fiscal years. Because the 
Department does not track the funding by program, but by activity, we 
did not audit the financial information provided by Health Canada. 
We reviewed the information for consistency with information that we 
reported on in our 2004 and 2006 chapters. We also interviewed key 
officials and we ensured the reasonableness of the financial 
information provided by Health Canada. 


6.28 We found that Health Canada allocated $10.1 million toward 
the direct costs of the program in the 2009-10 fiscal year (Exhibit 6.2), 
as compared to $2.7 million in the 2002-03 fiscal year (financial 
information presented in our 2004 report). Despite this increase, the 
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are not directly related to the service activity, 
for example, the cost of office space. 
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program continues to face challenges in meeting its responsibilities, 
as discussed in the next section. 


Exhibit 6.2 Total funding for pre-market and post-market activities in the 2008-09 and 2009-10 fiscal 
years ($ millions) 


Health Canada 
funding 
Direct costs 4.9 ‘Bye O73 5.0 Bel 10.1 


Indirect costs 


Industry funding 


User fees 


Source: Health Canada (unaudited) 


Health Canada is not meeting its obligations for the timely review of medical 
devices applications 


6.29. According to the Cabinet Directive on Streamlining 
Regulations, “departments and agencies are responsible for putting in 
place the processes to implement regulatory programs and to manage 
human and financial resources effectively, including... planning for 
the necessary human and financial resources . . .” The 2007-2012 
Medical Devices Strategic Plan describes one of its strategic objectives 
as having stable and adequate funding in place to meet all program 
requirements. 


6.30 In our 2006 audit, we reported that program managers were 
concerned that some activities were insufficient to meet their 
obligations. We noted that Health Canada had no program baseline— 
that is, it had not determined the required level of activities that the 
Medical Devices Program must carry out to meet its regulatory 
responsibility of protecting the health and safety of Canadians, and had 
not identified performance targets for these activities and the resources 
needed to do this work. We recommended that it do so and the 
Department agreed to begin a review of the funding allocated to 

the program and to set a baseline by 31 March 2008. 
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6.31 We examined reviews undertaken by Health Canada to address 
our recommendation, as well as its operational plans and performance 
reports. We also interviewed key officials. 


6.32 We found that Health Canada has established program baselines 
for its core pre-market activities, such as the review of applications for 
establishment licences and for medical devices licences. In particular, 
Health Canada has established the required level of activity to issue 
licences, and, in consultation with industry, has set service standards. 
However, Health Canada is not meeting these service standards for 
reviewing medical device applications (paragraphs 6.48 to 6.55). 
According to Health Canada, this is due to an estimated $4.2 million 
funding shortfall for the 2009-10 fiscal year. Because service standards 
for the review of medical device applications are not consistently met, 
we conclude that Health Canada is not meeting its obligations for this 
key activity. Our assessment of progress made on pre-market activities 


is found in Exhibit 6.7 (page 23). 


Health Canada does not know if it is meeting its obligations for its post-market 
activities 

6.33. We reported in our 2006 audit that program managers were 
concerned that some activities were insufficient to meet the Medical 
Devices Program’s regulatory responsibilities. We examined the 
following post-market activities to determine if program baselines had 
been established: 


* Inspections of establishments—on-site visits to assess how well an 
establishment conforms to sections of the Act and Regulations 
that mostly deal with either the safety or efficacy of medical 
devices or the administration of procedures designed to detect 
potential risks and respond when risks are identified. 


* Review of incident reports—reviewing documentation of device- 
related incidents received from manufacturers, health care 
professionals, and the public, to assess the underlying cause and 


seriousness of the incident. 


* Signal assessments and surveillance—continuous review of a 
variety of information sources, such as clinical and scientific 
literature and incident reports from foreign authorities, to assess 
and confirm possible connections (signals) between reported 
incidents and medical devices already on the market. 


* Risk communications—to inform the public, industry, and health 
care professionals about risks associated with the use of medical 
devices on the market, using different communication tools, such 
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as public advisories and warnings, letters to health care 
professionals, notices to hospitals, and foreign product alerts. 


Health Canada and the industry share the responsibility to 
communicate safety information about medical devices. 


6.34. We examined operational plans and performance reports for 
the 2008-09 and 2009-10 fiscal years for the post-market activities 
included in the scope of the audit. We also interviewed key officials. 


6.35 We found no complete baseline for any of the four examined 
post-market activities. In particular, Health Canada has not set the 
required level of effort for most of its post-market activities. For 
example, the Department has not determined how often 
establishments should be inspected to mitigate the identified risks. 
Therefore, it does not know whether it is doing too many or not 
enough inspections in a given year. 


6.36 Furthermore, except for signal assessment and surveillance, 
performance targets have not been set for the other three post-market 
activities we examined. No performance targets exist for inspections 
of establishments, the review of incidence reports, or risk 
communications. Although it is more complex to set targets for 

risk communications because of the participation of industry and 
departmental staff involved in the process, it is quite reasonable 

and feasible to set performance targets for these three post-market 
activities. 


6.37. Health Canada identified an estimated funding shortfall of 
$26.3 million for post-market activities in the 2009-10 fiscal year. But 
without a complete program baseline, the Department does not know 
if too much or not enough resources are allocated to these activities. 


6.38 We conclude that the Department does not know if it is 

carrying out the required level of post-market activity and that it has 
not established a desired level of performance. In other words, the 
Department does not know if it is meeting its obligations for post-market 
activities. We therefore assess progress on our past recommendation 
related to establishing program baselines as unsatisfactory (Exhibit 6.3). 


6.39 Recommendation. Health Canada should complete program 
baselines for post-market activities by defining the level of activities 
necessary to meet its obligation of protecting the health and safety of 
Canadians. It should also identify performance targets for these 
activities and the resources it needs to do this work. 
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The Department’s response. Agreed. Health Canada is committed to 
completing the establishment of program baselines. In May 2010, the 
Department began evaluating the international landscape of medical 
device inspections as a first step toward determining an appropriate risk- 
based inspection cycle. As a result, the Department expects to establish 
and implement a more risk-based inspection cycle in the 2012-13 fiscal 
year. 


In an effort to develop performance measures and baselines, Health 
Canada began reviewing the process of compliance verification 
activities in October 2010. The Department expects to establish 
appropriate baselines by the 2012-13 fiscal year. 


The Department has already begun developing performance standards 
and targets for its post-market surveillance review-related activities 
and risk communications activities in 2009 and expects full 
implementation by the 2011-12 fiscal year. A plan to enhance 

the effectiveness of risk communications is being developed for 
implementation during the 2011-12 fiscal year. 


Exhibit 6.3 Progress in addressing our recommendation on establishing program baselines 


Health Canada should establish program baselines (for the Medical 
Devices Program) by defining the required level of activities that 
the programs must carry out in order to meet their regulatory 
responsibilities of protecting the health and safety of Canadians. 
It should also identify the targeted performance for these activities, 


| Unsatisfactory 
and the resources needed to do this work. 


(See recommendation 8.27 of the 2006 November Report of 
the Auditor General of Canada, Chapter 8, Allocating Funds to 
Regulatory Programs—Health Canada) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Steps have been taken to address remaining funding shortfalls 


6.40 In 2006, we recommended that Health Canada develop a 
reporting mechanism that would require program managers to inform 
the departmental Executive Committee systematically about 
significant funding shortfalls and their effects. The Department agreed 
with our recommendation and stated that directorates were now 
reporting program results and their regulatory responsibilities to the 
Branch Executive Committee. 
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Full costs—the total cost of all resources used 
or consumed in delivering a program, including 
direct costs (such as salaries and employee 
benefit plans) and indirect costs (such as office 
space, information technology services, and 
human resource services). 
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6.41 We examined operational plans and performance reports for 
the 2008-09 and 2009-10 fiscal years and interviewed key officials. 
Program managers estimated a $4.2-million funding shortfall for 
pre-market activities and a $26.3-million funding shortfall for 
post-market activities for the 2009-2010 fiscal year. These shortfalls 
were discussed at the departmental level. 


6.42 ‘To address the long-standing funding shortfall for the Medical 
Devices Program, Health Canada began a project in 2004 to update its 
user fees. User fees had been established in 1996 and the cost to 
perform the regulatory services had increased since then. In May 2010, 
Parliament approved a proposal by Health Canada to increase user fees 
related to the evaluation, approval, and monitoring of medical devices. 
The new user fees will be governed by the User Fees Act, which 
provides for financial penalties to the regulator (Health Canada) if it 
does not meet service standards. (A key penalty for late processing of 
applications is that users may be granted fee reductions of up to 

50 percent.) Health Canada has yet to amend the regulations before 
the new fees can come into force. New user fees are expected to be 
implemented in the 2011-12 fiscal year. 


6.43 ‘To update user fees, Health Canada had to establish the 

full costs of the program. We examined documents produced by the 
Department that supported the Proposal to Parliament to increase user 
fees. We did not audit Health Canada’s analyses of these costs. 


6.44 According to Health Canada, it would cost $61.1 million to 
deliver the program in such a way that all obligations would be met. 
This amount includes the full costs of the program, namely 

$30.6 million for the 2009-10 fiscal year (Exhibit 6.2), as well as the 
estimated funding shortfalls of $4.2 million and $26.3 million that 
Health Canada has identified (paragraphs 6.32 and 6.37). 


6.45 If the user fee proposal is implemented as proposed, Health 
Canada estimates that revenues will increase by about 160 percent, 
going from $10.8 million in the 2009-10 fiscal year to $27.9 million 
in the 2011-12 fiscal year. On that basis, industry would be covering 
about 46 percent of the total program cost of $61.1 million. Health 
Canada’s contribution would increase from $19.8 million in the 
2009-10 fiscal year to $31.6 million in 2011-12. Health Canada has 
stated that the additional $12 million required in the 2011-12 fiscal 
year to deliver the program will be achieved through a combination 
of cost reductions, program efficiencies, and possible internal 
reallocations of funds. 
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6.46 In 2006, we recommended that Health Canada, in consultation 
with clients, set user fees for medical devices program services that are 
based on clear and measurable service standards. As part of the user 
fee proposal, Health Canada developed new standards, in consultation 
with industry, that are measurable. Health Canada indicated that it 
intends to publish guidance that will clarify the interpretation of these 
new service standards. 


6.47 We therefore assess the progress made on our past 
recommendations related to funding as being satisfactory (Exhibit 6.4). 


Exhibit 6.4 Progress in addressing our 2004 and 2006 recommendations on funding 


Health Canada should resolve the problems in its cost recovery 
program. It should determine the actual costs of the program and 
set fees based on those costs. 


(Recommendation 2.111 of the 2004 March Report of the Auditor 
General of Canada, Chapter 2, Health Canada—Regulation of 
Medical Devices) 


Satisfactory 


Health Canada should review the core funding allocated to [the 

Medical Devices Program] to ensure that it receives sufficient 

funding to meet regulatory responsibilities and to adequately 

protect the health and safety of Canadians. Satisfactory 


(See recommendation 8.43 of the 2006 November Report of 
the Auditor General of Canada, Chapter 8, Allocating Funds to 
Regulatory Programs—Health Canada) 


Health Canada should, in consultation with clients, set user fees 
for Medical Devices program services that are based on clear and 
measurable service standards. Satisfactory 


(See recommendation 8.46 of the 2006 November Report of 
the Auditor General of Canada, Chapter 8) 


Health Canada should develop a reporting mechanism that would 
require program managers to inform the departmental Executive 
Committee systematically about 


the level of financial resources allocated to the Medical Devices 
Program, 


results achieved with the funds, and Satisfactory 


extent to which the program has met its regulatory 
responsibilities, including significant gaps and their 
consequences. 


(See recommendation 8.84 of the 2006 November Report of 
the Auditor General of Canada, Chapter 8) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 
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Health Canada’s decisions on medical device applications are not timely 


6.48 Health Canada conducts pre-market activities before it 
authorizes a company to market a medical device. These activities 
include reviewing submissions of scientific evidence by manufacturers 
to determine the device’s safety and efficacy. 


6.49 In 2004, we found that the pre-market evaluation process was 
consistently followed but there were delays in the authorizing of 
licences. We recommended that Health Canada ensure that 
Canadians have timely access to all available safe and effective devices. 
The Department agreed with our recommendation and stated that it 
would undertake a review to determine the reason(s) it was not 
meeting performance targets. It committed to begin taking corrective 
action in spring 2005. 


6.50 The User Fees Act states that, before a regulating authority fixes, 
increases, or expands the application of a user fee or increases its 
duration, it must establish standards against which the performance of 
the regulating authority can be measured. 


6.51 Both establishment licences and medical device licences are 
subject to user fees. As part of establishing its user fees, Health Canada 
had consulted with the Canadian medical device industry and 
established service standards (Exhibit 6.5). 


Exhibit 6.5 Health Canada consulted with the Canadian medical devices industry 
to set service standards 


CATO pple 5 Nae Wy ea Fe 


Auhenbing device licences 
Class II: review time to first decision* 15 calendar days 
Class Ill: review time to first decision 75 calendar days 90% 
Class IV: review time to first decision 90 calendar days 


*{ first decision can either be a decision to request additional information from the applicant or 
a final decision to either approve or deny the application for a medical device licence. 


6.52 To determine the timeliness of Health Canada’s decisions to 
approve or deny a medical device licence, we examined whether the 
Department was meeting its established service standards for its review 
of Class II, III, and IV medical device licence applications. We also 
examined management reports related to program performance for the 
period under audit, analyzed the database on licences issued, and 
interviewed key officials. 
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6.53 We found that the Department is not consistently meeting its 
service standards related to applications for medical device licences. 
For example, first decisions were made within the 90-day service 
standard for only 55 percent of Class IV medical device applications in 
the 2009-10 fiscal year. As well, our analysis of the number of days to 
review a Class IV medical device application in the 2009-10 fiscal year 
showed that 45 percent of these applications required longer than the 
90-day service standard, by an average of 63 days. 


6.54 Meeting service standards for medical devices has been a long- 
standing issue for Health Canada. In our 2004 audit, we reported that, 
in 2002, an estimated 42 percent of Class III medical devices were 
processed within the service standard of 75 days, and that an estimated 
31 percent of Class IV medical devices were processed within the 
service standard of 90 days. The performance reflected in Exhibit 6.6 
for the 2008—09 and 2009-10 fiscal years shows that meeting service 
standards continues to be a problem. 


Exhibit 6.6 Health Canada has consistently fallen short of its service standards for 
reviewing applications 


Percentage of licence applications 


100 
58% % 69% | | 63% Licence applications 
90 reviewed within 
80 service standards 
+ Fe [_] 2002 
. #5 BS 2008-2009 
vai 


50 BB 2009-2010 
40 Licence applications 
not reviewed 
30 [_] Percentage of 
20 applications that were 
not reviewed within 
10 the service standard 
0 L =. 
Class Il Class Ill Class IV 
(15 days) (75 days) (90 days) 


Classes of medical devices and service standards 


* Data from 2002 was collected during our 2004 audit. At that time, we did not collect data on 
applications for Class || medical devices. The percentages for Class II! and lV applications were estimated 
from a random sample of applications. 


6.55 Health Canada indicated that factors contributing to the delays 
in reviews include the rapid growth of medical device technology with 
more products using, for example, nanotechnology. This means that 
more hours are needed to complete a review. According to Health 
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Canada officials, other factors contributing to delays in reviews include 
reviewers having to complete tasks other than review work. These 
factors include responding to regulatory questions, information 
technology that is outdated, lack of funding and staff, and an increased 
number of requests for additional information because applications are 
incomplete. The case study on Health Canada’s decision on an 
application for an automated blood collection system shows that 
service was not timely. 


Case Study—Automated Blood Collection System 


The automated red cell collection system is a device that allows blood collectors to 
collect two units of red cells from a single donor instead of one unit. Health Canada 
received an application from an American company for this Class II] device on 

14 September 2007. Although the service standards indicate that a first review for a 
Class Ill device application must be processed within 75 days, the first review took 
more than 7 months (221 days), which is almost 5 months (146 days) more than the 
service standard. At the end of this first review, Health Canada requested additional 
information to the manufacturer. It took more than three weeks for the manufacturer to 
respond to the request. It then took Health Canada more than two months to conduct 
its second review and to reach its final decision to approve the device. Almost 

11 months (333 days) had elapsed since the date the application was submitted. 


Untimely decisions can have important consequences 


6.56 Both the Canadian public and health care professionals benefit 
from having timely access to safe and effective medical devices. While 
it is important that Health Canada take the necessary time to properly 
evaluate a device for safety and therapeutic effectiveness, delays in 
authorizing licences means Canadians’ access to the health benefit of 
these devices is delayed. 


6.57 ‘To understand the consequences of untimely decisions for 
industry, we surveyed 77 US manufacturers who, between April 2008 
and March 2010, had been issued a licence for a new Class III or IV 
medical device by Health Canada. We sent the survey only to US 
manufacturers because they represent almost half of all licences issued 
by Health Canada for Class II] and IV medical devices. Of the 

77 companies surveyed, we received 52 responses (a response rate of 
68 percent). 


6.58 Of the 52 responses we received, 28 were from companies whose 
applications had not been approved within the service standard. Of 
these, 17 told us that the consequences for them of Health Canada’s 
delayed approval included delayed launches of their devices to the 
public and loss of sales. Although not specifically mentioned in the 
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responses, delayed approvals also mean that Canadians’ access to the 
health benefit of these devices is delayed. 


6.59 Untimely decisions to approve or deny a medical device licence 
may also have financial consequences for Health Canada. The User 
Fees Act states that, before a regulating authority fixes, increases, or 
expands the application of a user fee or increases its duration, it must 
establish standards against which the performance of the regulating 
authority can be measured. When new user fees come into effect, the 
regulator (in this case, Health Canada) may suffer financial 
penalties—fee reductions of up to 50 percent for users—if it fails to 
meet service standards. 


6.60 Recommendation. Health Canada should ensure that it meets 
service standards for the review of medical device applications so that 
Canadians and health care professionals have timely access to safe and 
effective devices. 


The Department’s response. Agreed. Health Canada has received 
Parliamentary approval to increase user fees for medical devices. Once 
implemented, these fees will contribute to more stable funding for key 
regulatory services such as product reviews, post-market surveillance, 
and inspections. Increased revenues will be reinvested into specific 
activities for which the fees were collected. These revenues will allow 
Health Canada to meet its well-established and internationally 
recognized performance standards. The Department anticipates 
meeting the service standards associated with the Cost Recovery 
Proposal by April 2012. 


Health Canada could benefit from increased international cooperation for its 
pre-market activities 


6.61 According to the Cabinet Directive on Streamlining 
Regulations, departments and agencies are to take advantage of 
opportunities for cooperation, either bilaterally or multilaterally. Our 
2004 Report included a recommendation, accepted by Health Canada, 
that the Department consider engaging in activities that would allow it 
to benefit from international regulatory cooperation for its pre-market 
activities. For this audit, we examined departmental documentation 


related to international activities. We also interviewed key officials at 
Health Canada. 


6.62 Health Canada’s 2007 Blueprint for Renewal II commits the 
Department to enhanced international cooperation by focusing 
resources on working within existing agreements to improve regulatory 
performance. This includes activities that would support Health 
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Canada’s decision making, for example, the sharing of real-time 
information on risk assessments and product reviews. 


6.63 Health Canada recognizes the benefits of making direct use of 
foreign reviews, such as 


* drawing on expertise, held by another jurisdiction, in a subject 
area; 


¢ better efficiency in the application review process; and 


¢ better resource management, including the reallocation of 
resources to priority or complex applications. 


6.64 ‘To assess the potential benefits of international cooperation, 

we asked the companies we surveyed to tell us whether the device for 
which they sought Health Canada’s approval had been previously 
approved by Europe, the United States, Australia, or Japan; and, if so, 
when that approval had been granted. We did not compare Health 
Canada’s review time to that of other jurisdictions. Of the 52 responses 
we received from US companies, 42 stated that the medical device for 
which they sought Health Canada’s approval had already been 
approved in one or more of the listed jurisdictions. Devices had been 
on the market in other jurisdictions for an average of two years before 
companies applied to Health Canada for approval. 


6.65 Therefore, our survey results show that opportunities exist 

for Health Canada to make use of assessment work done in other 
jurisdictions in its own assessment of the safety and efficacy of medical 
devices with a view to increase the efficiency of its processes. 


6.66 We found that, so far, Health Canada’s international strategies 
have mainly focused on the harmonization of medical devices 
regulations, an area the Department has been involved in for almost 
20 years. The Department is a founding member of the Global 
Harmonization Task Force (GHTF), created in 1992 to respond to the 
growing need for international harmonization of the regulations. The 
GHTF is a voluntary group of regulatory officials and industry 
representatives from the European Union, the United States, Canada, 
Australia, and Japan and from non-governmental organizations. 


6.67 Health Canada has had some success in cooperating with one 
jurisdiction. Since 2004, Health Canada has been working with the 
Australian government’s Therapeutic Goods Administration on 

the reciprocal recognition of quality system audits for medical device 
manufacturers. As of October 2010, both regulators are able to accept 
each other’s audit results. Because this initiative is so new, 
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Health Canada has not yet received Australia’s recent audit results, 
so efficiencies have not yet been realized. 


6.68 Although Health Canada has been able to cooperate with other 
jurisdictions in other areas of importance, we found that strategies 
aimed at making direct use of foreign reviews to enhance the efficiency 
of Health Canada’s medical device review process were only recently 
developed—six years after our 2004 recommendation related to this 
was accepted by the Department. In June 2010, the Department 
approved a Master Plan for the Use of Foreign Regulatory Information 
in the Regulatory Review of Health Products. This plan’s objectives 
are to enhance the efficiency of the Department’s review process and 
to improve Canadians’ timely access to devices that are safe and of 
high quality. 


6.69 As of October 2010, the Department had developed guidance on 
the extent to which foreign reviews can be used. It also developed an 
approach to enhance participation on foreign regulatory scientific 
committees. Health Canada officials told us that a pilot project on the 
use of foreign reviews should be implemented in the 201 1—12 fiscal 
year. 


6.70 Health Canada has recognized that there are challenges in 
realizing the potential of international cooperation, and we concur. 
Such challenges include 


* the availability and scientific quality of foreign review reports, 


* foreign regulatory authorities’ legal frameworks enabling the 
sharing of complete reviews, 


* the lack of legal authority in Canada to compel industry to 
provide foreign reviews when submitting an application, and 


* maintaining the integrity of Health Canada’s scientific capacity 
while making more systematic use of foreign reviews. 


Although these are significant challenges, Health Canada recognizes 
that there are advantages to increasing the level of international 
regulatory cooperation. 


6.71 Because initiatives that could improve the efficiency of Health 
Canada’s review process are recent and have not yet had a positive 
impact on regulatory performance, we conclude that Health Canada 
has made insufficient progress in its efforts to work cooperatively with 
international jurisdictions to leverage resources, information, and 
knowledge related to pre-market assessments of medical devices 
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(Exhibit 6.7). Health Canada is not making direct use of foreign 
reviews that could lead to program efficiencies. 


6.72 Recommendation. Health Canada should implement the 
Master Plan for the Use of Foreign Regulatory Information, which it 
developed in 2010, for international cooperation at the pre-market 
stage, with a view to improving regulatory performance by making use 
of resources, information, and knowledge from other jurisdictions. 


The Department’s response. Agreed. Health Canada has begun 
implementing a three-year plan to make more efficient, consistent, and 
systematic use of international regulatory information for pre-market 
assessments and expects to have the plan fully operational by 

March 2013. The Department will continue collaborating with 
international counterparts to allow timely and complete access to 
information from other jurisdictions. 


Exhibit 6.7 Progress in addressing our recommendations for pre-market activities 


Health Canada should ensure that Canadians have timely access 
to all available safe and effective devices. More specifically, it 
should ensure that all devices are approved within the 
performance targets and should consider engaging in activities that 
would allow it to benefit from international regulatory co-operation. Unsatisfactory 


(Recommendation 2.75 of the 2004 March Report of the Auditor 
General of Canada, Chapter 2, Health Canada—Regulation of 
Medical Devices) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Managing risks from devices Health Canada has significantly expanded its post-market activities 


available in Canada ; 
6.73 Post-market activities are conducted once a device has come 


onto the Canadian market. They focus on product safety and 
effectiveness, and regulatory compliance. We examined the following 
post-market activities: 


* inspections of establishments, 
* the review of incident reports, 
* signal assessment and surveillance, and 


° risk communication. 
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6.74. We noted in our 2004 audit that health and safety risks at the 
post-market stage needed to be better managed. Health Canada agreed 
to review its (then) current level of post-market activities and to 
develop an action plan to address our findings by spring 2005. 


6.75 To determine whether the Department adequately managed the 
risks related to medical devices available on the Canadian market, we 
examined the steps taken by the Department to allocate resources to 
devices presenting the greatest risk. We reviewed management reports 
related to program performance for the period under audit and we 
interviewed program officials. As mentioned in paragraph 6.27, we did 
not audit the financial information provided by Health Canada. 


6.76 We found that the Department has significantly increased the 
level of funding for the direct costs of its post-market activities 

since 2004. In our 2004 report, we observed that Health Canada 
allocated close to $3 million to its post-market activities, including 
revenues from user fees. In the 2009-10 fiscal year, the Department 
allocated $11.8 million to its post-market activities, including user fees, 
which represents a 293 percent increase. 


6.77 In 2004, the Department established a national inspection 
program to assess the industry’s compliance with the Food and Drugs 
Act and the Medical Devices Regulations. Health Canada stated that the 
number of establishments it inspected annually has steadily increased 
since the introduction of the program to reach 226 inspections in the 
2009-10 fiscal year, or 14 percent of the licensed establishments 
located in Canada. 


6.78 We noted that the number of reports on device-related incidents 
received from manufacturers, health care professionals, or the public 
has increased since our last audit. In 2004, we reported that Health 
Canada received 1,000 incident reports in the 2001-02 fiscal year. The 
Department has stated that the number rose to 4,867 in the 2009-10 
fiscal year, an increase of 387 percent. In 2009, the Department 
established, on a pilot basis, a network involving 10 health care 
facilities across Canada to further improve the number and quality of 
incident reports collected. As part of this pilot, Health Canada obtains 
information on incidents from each of the participating facilities. 
During the first year of the pilot, these sites prepared 161 incident 
reports. 


6.79 The Department has also increased its capacity to carry out 
surveillance activities and signal assessments (paragraph 6.33). 
In 2004, we reported that Health Canada had no proactive system for 
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identifying patterns in incident reports or complaints that could signal 
a serious safety concern. Health Canada has since developed its 
capacity to proactively identify safety risks and take steps to address 
them. In the 2009-10 fiscal year, Health Canada told us that it 
detected 49 signals requiring additional analysis and did an in-depth 
assessment of 17. 


Health Canada identifies risks for key post-market activities but does not know if 
they are adequately mitigated 


6.80 The Health Products and Food Branch mandate includes 
“minimizing health risk factors to Canadians while maximizing the 
safety provided by the regulatory system for health products and food.” 
We examined whether Health Canada allocates its resources to 
medical devices that present the greatest risk to the health and safety 
of Canadians. 


6.81 One of the key strategic actions identified in the Health 
Products and Food Branch Strategic Plan 2007—2012 is to strengthen 
post-market activities, which includes surveillance and monitoring of 
medical devices’ safety and therapeutic effectiveness. In addition, two 
objectives of Health Canada’s Blueprint for Renewal II are to ensure 
that regulatory scrutiny is consistent with the risks posed by a product 
and to develop a stronger post-market surveillance system. 


6.82 Inspection of establishments. We reviewed Health Canada’s 
inspection strategy, which has been in force since 2003, and the annual 
inspection plans for the 2008-09 and 2009-10 fiscal years, as well as 
guidelines for carrying out inspections and management reports on 
performance. We also interviewed program officials. 


6.83 According to Health Canada, there were 1,624 companies in 
Canada subject to inspection in the 2009-10 fiscal year. Because 
the Medical Devices Regulations place significant responsibility for 
protecting the health and safety of Canadians on the industry itself, 
it is particularly important that Health Canada assess its activities. 
Since it is not feasible for Health Canada to inspect every 
establishment each year, the Department has adopted a risk-based 
approach for this activity. 


6.84 Annual inspection plans identify the establishments that pose 
the greatest risk of non-compliance with the Food and Drugs Act and 
Medical Devices Regulations. For 2009-10, these are 


¢ Class I manufacturers; 


¢ Class III and IV importers; 
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* manufacturers of house brands (for example, store brand 
toothbrushes); and 


¢ establishments that have had an establishment licence for more 
than five years without an inspection. 


Because manufacturers of Class II, III, and IV devices are subject to 
quality systems audit, they have been excluded from the risk-based 
inspection plans. However, Health Canada can still inspect these 
manufacturers when warranted. 


6.85 Once the highest-risk establishments have been identified 
through the risk-based inspection plan, it is up to Health Canada’s 
regional offices to select the establishments and carry out the actual 
inspections. Health Canada prepares quarterly reports that present 
results achieved through inspections. However, the Department does 
not compile information from regional offices on the number of 
establishments inspected by risk criteria outlined in the inspection 
plan. This means that Health Canada does not know if the risks 
identified in the yearly inspection plan are being adequately mitigated 
and whether resources are being allocated to devices that present the 
greatest risk. 


6.86 We found that the 2009-10 risk-based inspection plan did not 
consider the 523 licensed establishments identified by Health Canada 
as being located outside of Canada when it determined which 
establishments posed the greatest risk. Establishments located outside 
of Canada represent 24 percent of the 2,147 licence holders. This 
means that Health Canada has no information on the risks posed by 
these establishments. Indeed, inspection activities have always 
excluded companies located outside of Canada. The strategy, 
developed in 2003, advises inspectors to contact companies located 
outside of Canada to ask for and review information and records and 
to assess compliance with the regulations. To date, this has not been 
done. Health Canada has recently stated that it will develop a pilot 
project in the 2010-11 fiscal year to inspect foreign establishments. 


6.87. Recommendation. Health Canada should assess the risks posed 
by foreign establishments and consider the results of this assessment 
when it develops the annual risk-based inspection plan and carries out 
the inspections to ensure that the risks are adequately addressed. 


The Department’s response. Agreed. Health Canada identified this 
area as a priority in its 2010-2011 inspection operational plan and has 
already begun preliminary work in this area. The Department expects 
to conduct a risk assessment of foreign establishment licence holders 


Status Report of the Auditor General of Canada—2011 


Status Report of the Auditor General of Canada—2011 


REGULATING MEDICAL DEVicEs—HEALTH CANADA 


by the end of 2011-12. Any further activities may require regulatory 
amendments and will depend on both the results of the above risk 
assessments and an analysis of existing legislation. 


6.88 Incident reports. Health Canada developed criteria to assess 
incidents, which include the 


* likelihood of similar incidents; 

. populedon at risk; 

* benefits from the device; 

* magnitude of public and professional concern; 


* user qualifications (professional, trained user versus unqualified, 
untrained user); 


* potential effectiveness of regulatory action; and 


* user awareness of the problem. 


6.89 However, we found that Health Canada does not assess whether 
incident reports on specific devices have been reviewed according to 
the risks identified for that device or that incident reports are reviewed 
in a timely manner. Without this information, it is difficult for Health 
Canada to know if the risks identified are being adequately mitigated 
and whether resources are being allocated to devices that present the 
greatest risk. 


6.90 Recommendation. Health Canada should regularly assess 
whether resources have been allocated to the identified risks for 
inspections and the review of incident reports and whether these risks 
have been adequately mitigated. 


The Department’s response. Agreed. Health Canada will continue to 
assess whether resources have been allocated to identify risks by 
determining an appropriate risk-based inspection cycle and 
establishing appropriate baselines as described in the Department's 
response to Recommendation 6.39. In addition, the Department will 
evaluate incident reporting through the Sentinel system and establish 
performance standards consistent with other product lines by the 
2011-12 fiscal year. Data from medical device incident reports, other 
post-market activities, and associated performance measures will be 
used to identify high-risk areas requiring risk mitigation activities. The 
Department will also endeavour to determine, to the extent possible, 
whether the identified risks have been mitigated. 
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6.91 Signal assessment and surveillance. We found that Health 
Canada has a process in place to assess risks for the signals it detects, 
based on established criteria. These criteria include death, life 
threatening event, hospitalization (required or prolonged), disability, 
or incapacity caused by or related to the use of a medical device. 
Resources are then allocated to the higher risk signals. 


6.92 Risk communication. In 2004, we reported that the 
Department had never developed a communications plan or 
communications strategy to inform the public, industry, and health 
care professionals about risks associated with the use of certain medical 
devices on the market. We found that, since our 2004 audit, Health 
Canada has developed a guide for risk communication activities 
involving medical devices. The guide describes the various risk 
communication tools that the Department and industry can employ. It 
also outlines what type of communication tool to use depending on the 
urgency of the situation and the targeted audience. Health Canada 
stated that when a risk communication is needed, it is considered a 
high priority and takes precedence over other tasks. This allows the 
Department to allocate its resources to areas of greatest risk within the 
Medical Devices Program. 


6.93 However, we also found that Health Canada has not assessed 
whether its risk communications for medical devices effectively reach 
their target audiences and do so in a timely manner. Without such 
assessments, it is difficult for Health Canada to determine if its risk 
communication activities for medical devices are timely or if they are 
reaching target audiences. 


6.94 Recommendation. Health Canada should assess whether its 
communications to inform the public, industry, and health care 
professionals about risks associated with medical devices on the market 
are effectively reaching their target audiences and doing so in a timely 
manner. 


The Department’s response. Agreed. Health Canada will undertake a 
systematic evaluation of the effectiveness of its risk communications 
for all product lines, including medical devices. The plan will identify 
current gaps and challenges with risk communication evaluation and 
will highlight short-term, medium-term, and long-term activities aimed 
at systematic evaluation of Health Canada’s risk communication tools. 
This plan will be initiated in the 2011-12 fiscal year with an 
anticipated completion date of the 2013-14 fiscal year. 
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Health Canada is working cooperatively with international jurisdictions on post- 
market activities but needs to determine if further actions are necessary 


6.95 Health Canada has stated that it intends to pursue opportunities 
to work cooperatively with international jurisdictions. We examined 
documents related to international cooperation to support Health 
Canada’s post-market activities for medical devices. We also 
interviewed program officials at Health Canada. 


6.96 We found that Health Canada is working with international 
jurisdictions and has updated a guidance document on incident 
reporting. This update is based on the work that the Department is 
doing with the Global Harmonization Task Force. In addition, the 
Department indicated that it makes use of foreign incident data to 
inform signal assessments and surveillance activities. It has also 
developed a post-market surveillance strategy called Planning for Our 
Future: Federal Regulatory Post-Market Surveillance Strategy 2007- 
2012. This strategy states that the Department would like to increase 
its capacity for systematically reviewing foreign incident reports. 
However, the strategy does not describe concrete actions that it needs 
to take to do so, with accompanying timelines.and resources. 
Furthermore, the Department has not determined whether further 
actions are necessary for other key post-market activities such as 
inspections, signal assessments, and risk communications. 


6.97 We conclude that Health Canada is working cooperatively with 
international jurisdictions on post-market activities by leveraging 
resources, information, and knowledge on incident reports, but it 
needs to determine if further actions are necessary. 


6.98 Recommendation. Health Canada should determine whether it 
needs to take additional steps to increase international cooperation for 
its post-market activities. If so, the Department should identify 
concrete actions to be taken, along with timelines and needed 
resources. 


The Department’s response. Agreed. Over the next three years, 
Health Canada will identify gaps in international cooperation for post- 
market activities and the steps needed to increase international 
collaboration for post-market activities such as inspections, signal 
assessments, and communications. A three-year implementation work 


plan (2010-2013) will be developed. 
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6.99 In summary. Overall, we found that Health Canada has made 
satisfactory progress in implementing our past recommendation on 
post-market activities. The Department has established a national 
inspection program and has increased its capacity to analyze post-market 
surveillance information and incident reports (Exhibit 6.8). Health 
Canada allocates its resources to medical devices that pose the greatest 
risk for signal assessment and surveillance. However, the Department 
has yet to determine whether the risks associated with medical devices 
available on the Canadian market (that establishment inspections and 
the review of incident reports are designed to address) have been 
adequately mitigated, and whether its risk communications reach the 
target audiences and do so in a timely manner. Therefore, Health 
Canada is not adequately managing the risks related to medical devices 
available on the Canadian market for these three post-market activities. 


Exhibit 6.8 Progress in addressing our recommendations on post-market activities 


Health Canada should ensure that it manages the risks and 
benefits related to medical devices after they are available for sale. 
More specifically, it should 


* engage in active inspection of industry operations at the post- 
market phase to verify compliance with the Regulations; 


* improve the collection, analysis, and interpretation of post- 
market surveillance information by increasing the number and 
quality of adverse events reports collected and by improving its 
analytical and interpretive capability; and 


Satisfactory 


* improve its approach to communication of safety concerns so 
that those who need to know are informed in a timely manner. 


(See recommendation 2.100 of the 2004 March Report of the 
Auditor General of Canada, Chapter 2, Health Canada—Regulation 
of Medical Devices) 


Satisfactory—Progress is satisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 


Unsatisfactory—Progress is unsatisfactory, given the significance and complexity of the issue, 
and the time that has elapsed since the recommendation was made. 
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Conclusion 


6.100 Overall, we concluded that Health Canada has not made 
satisfactory progress in acting on commitments it made in response to 
recommendations from our 2004 and 2006 reports. While overall 
program funding has increased and program capacity has been 
enhanced, primarily at the post-market stage, the Department is not 
fully meeting its obligations in relation to medical devices. 


6.101 Decisions on applications for medical device licences are not 
delivered in a consistently timely way. As a result, the service standards 
agreed to between the Department and industry are often not met. 
This delays access to these devices by Canadians and health care 
professionals. 


6.102 Health Canada has significantly expanded its post-market 
activities by, for example, establishing a national inspection program, 
and has identified risks associated with medical devices already 
available on the Canadian market. It has yet to determine whether the 
risks that the establishment inspections and the review of incident — 
reports are designed to address have been adequately mitigated, and 
whether its risk communications reach the target audiences and do so 
in a timely manner. Therefore, Health Canada is not adequately 
managing the risks related to medical devices available on the 
Canadian market for these three post-market activities. 


6.103 Greater cooperation with foreign regulatory bodies has the 
potential to improve the Medical Devices Program’s regulatory 
performance. The Department’s work to date on the international 
front has not taken advantage of foreign resources, information, 
and knowledge at the pre-market stage. 
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About the Audit 


All of the audit work in this chapter was conducted in accordance with the standards for assurance 
engagements set by The Canadian Institute of Chartered Accountants. While the Office adopts these 
standards as the minimum requirement for our audits, we also draw upon the standards and practices of 
other disciplines. 


Objectives 


The overall audit objective was to determine whether Health Canada has made satisfactory progress in 
implementing selected recommendations from previous reports from the Auditor General of Canada about 
adequately managing the Medical Devices Program. 


The audit sub-objectives were to determine whether the Department 
* makes timely decisions about allowing medical devices to be sold in Canada, 
+ adequately manages the risks related to medical devices available on the Canadian market, and 


* meets its obligations for medical devices. 


Scope and approach 


Our audit focused on the three directorates involved in the delivery of the Medical Devices Program 
and examined whether selected previous recommendations associated with activities undertaken at 
the pre-market and post-market stages and relating to program management have been implemented. 


At the pre-market stage, we examined the timeliness of reviews of medical device applications, and the 
steps taken to increase program efficiency through international regulatory cooperation with key 
regulatory agencies in such countries as the United States and Australia. As well, we randomly sampled 
100 market authorizations from 602 Class III and IV applications from US manufacturers. We did this to 
cover the period between April 2008 and March 2010, using the provision that the sample include no 
more than one submission per applicant. Seventy-seven of the sampled authorizations had valid applicant 
contact information and the corresponding applicants were surveyed. 


At the post-market stage, we examined the steps taken by the Department to allocate resources to devices 
presenting the greatest risk and to increase program efficiency through international regulatory 
cooperation. 


Finally, we reviewed departmental efforts to 
- determine required levels of activity, performance, and funding necessary to meet its obligations; 
* monitor program performance against such requirements; and 
* report on and address performance and funding gaps, as appropriate. 


We examined the funding allocated to the Medical Devices Program for the 2008-09 and 2009-10 fiscal 
years. Because the Department tracks the funding by activity, and not by program, we did not audit the 
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financial information provided by Health Canada. We reviewed the information for consistency with 
information presented in our 2004 and 2006 reports. We also interviewed key officials and we ensured the 
reasonableness of the financial information provided by Health Canada. 


We collected evidence through interviews with key Health Canada officials involved with the program at 
headquarters and in several regional offices. We met with representatives from the medical devices 
industry and the health care field. We reviewed documentation, including regulations, operational plans, 
performance reports, and program reviews. We also surveyed companies to which Health Canada had 
issued a licence for a new medical device in the 2008-09 and 2009-10 fiscal years. 


We did not examine the monitoring of investigational testing (clinical trials), unlicensed devices, and the 


reuse of single-use devices. As well, we did not audit the process of resource allocation reported in our 
2006 report. 


Criteria 


Health Canada reviews licence applications for medical devices 
in a timely manner. 


User Fees Act, section 4(2)(c) 


Policy on Management of Applications for Medical Device 
Licences and Investigational Testing Authorizations, Health 
Canada, 2001 


Health Canada works cooperatively with international Cabinet Directive on Streamlining Regulations, section 4.4, 


jurisdictions by leveraging resources, information, and Treasury Board, 2007 
knowledge related to pre-market assessments of medical 
devices assessments. 


Health Products and Food Branch 2007-12 Strategic Plan: 
Protecting and Promoting the Health and Safety of Canadians, 
Health Canada, 2007 


Blueprint for Renewal Il: Modernizing Canada’s Regulatory 
System for Health Products and Food, Health Canada, 2007 


Health Canada allocates its resources to medical devices that Cabinet Directive on Streamlining Regulations, section 4.5, 
present the greatest risk to the health and safety of Canadians. Treasury Board, 2007 


Regulation of Medical Devices, Auditor General of Canada, 
2004—Departmental response to recommendation 2.100 


Medical Devices Program Strategic Plan 2007-12: Building 
for the Future, Health Canada, 2007 


Health Products and Food Branch 2007-12 Strategic Plan: 
Protecting and Promoting the Health and Safety of Canadians, 
Health Canada, 2007 


Blueprint for Renewal II: Modernizing Canada’s Regulatory 
System for Health Products and Food, Health Canada, 2007 
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Health Canada works cooperatively with international 

- jurisdictions by leveraging resources, information, and 
knowledge on device-related incidents, inspections, signal 
assessments and surveillance, and risk communications. 


Cabinet Directive on Streamlining Regulations, section 4.4, 
Treasury Board, 2007 


Health Products and Food Branch 2007-12 Strategic Plan: 
Protecting and Promoting the Health and Safety of Canadians, 
Health Canada, 2007 


Blueprint for Renewal Il: Modernizing Canada’s Regulatory 
System for Health Products and Food, Health Canada, 2007 


Health Canada determines the required levels of activity, 
performance, and funding needed to meet its obligations. 


Cabinet Directive on Streamlining Regulations, section 4.5, 
Treasury Board, 2007 


Allocating Funds to Regulatory Programs, Auditor General of 
Canada, 2006—departmental response to 
recommendations 8.27 and 8.43 


Medical Devices Program Strategic Plan 2007-12: Building 
for the Future, Health Canada, 2007 


Health Canada meets the required levels of activity, 
performance, and funding needed to fulfill regulatory and 
program responsibilities. 


Cabinet Directive on Streamlining Regulations, section 4.6, 
Treasury Board, 2007 


Health Canada identifies and addresses performance and 
funding gaps/shortfalls, as needed. 


Cabinet Directive on Streamlining Regulations, section 4.6, 
Treasury Board, 2007 


Allocating Funds to Regulatory Programs, Auditor General of 
Canada, 2006—departmental response to 
recommendation 8.84 


Budget Management Framework Handbook, Health 
Canada, 2007 


A Manager's Guide to Operating Budgets, Treasury 
Board, 1992 


Management reviewed and accepted the suitability of the criteria used in the audit. 


Period covered by the audit 


The audit covered the period of 1 April 2008 to 31 March 2010. Audit work for this chapter was 


substantially completed on 29 October 2010. 
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Audit team 


Assistant Auditor General: Neil Maxwell 
Principal: Louise Dubé 
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For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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Appendix List of recommendations 


The following is a list of recommendations found in Chapter 6. The number in front of the 
recommendation indicates the paragraph where it appears in the chapter. The numbers in parentheses 
indicate the paragraphs where the topic is discussed. 


Funding for the Medical Devices Program 


6.39 Health Canada should complete 
program baselines for post-market 
activities by defining the level of 
activities necessary to meet its 
obligation of protecting the health.and 
safety of Canadians. It should also 
identify performance targets for these 
activities and the resources it needs to 


do this work. (6.33-6.38) 


Approving medical devices for sale in Canada 


6.60 Health Canada should ensure 
that it meets service standards for the 
review of medical device applications 
so that Canadians and health care 
professionals have timely access to safe 
and effective devices. (6.48-6.59) 


Chapter 6 


Agreed. Health Canada is committed to completing the 
establishment of program baselines. In May 2010, the Department 
began evaluating the international landscape of medical device 
inspections as a first step toward determining an appropriate risk- 
based inspection cycle. As a result, the Department expects to 


establish and implement a more risk-based inspection cycle in the 
2012-13 fiscal year. 


In an effort to develop performance measures and baselines, 
Health Canada began reviewing the process of compliance 
verification activities in October 2010. The Department expects 
to establish appropriate baselines by the 2012-13 fiscal year. 


The Department has already begun developing performance 
standards and targets for its post-market surveillance review- 
related activities and risk communications activities in 2009 and 
expects full implementation by the 2011-12 fiscal year. A plan to 
enhance the effectiveness of risk communications is being 
developed for implementation during the 2011-12 fiscal year. 


Agreed. Health Canada has received Parliamentary approval to 
increase user fees for medical devices. Once implemented, these 
fees will contribute to more stable funding for key regulatory 
services such as product reviews, post-market surveillance, and 
inspections. Increased revenues will be reinvested into specific 
activities for which the fees were collected. These revenues will 
allow Health Canada to meet its well-established and 
internationally recognized performance standards. The 
Department anticipates meeting the service standards associated 
with the Cost Recovery Proposal by April 2012. 
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6.72 Health Canada should 
implement the Master Plan for the Use 
_of Foreign Regulatory Information, 
which it developed in 2010, for 
international cooperation at the 
pre-market stage, with a view to 
improving regulatory performance by 
making use of resources, information, 
and knowledge from other jurisdictions. 


(6.61-6.71) 


Managing risks from devices available in Canada 


6.87 Health Canada should assess the 
risks posed by foreign establishments 
and consider the results of this 
assessment when it develops the annual 
risk-based inspection plan and carries 
out the inspections to ensure that the 


risks are adequately addressed. 
(6.82-6.86) 


6.90 Health Canada should regularly 
assess whether resources have been 
allocated to the identified risks for 
inspections and the review of incident 
reports and whether these risks have 
been adequately mitigated. 


(6.88-6.89) 


Agreed. Health Canada has begun implementing a three-year 
plan to make more efficient, consistent, and systematic use of 
international regulatory information for pre-market assessments 
and expects to have the plan fully operational by March 2013. 
The Department will continue collaborating-with international 
counterparts to allow timely and complete access to information 
from other jurisdictions. 


Agreed. Health Canada identified this area as a priority in its 
2010-2011 inspection operational plan and has already begun 
preliminary work in this area. The Department expects to 
conduct a risk assessment of foreign establishment licence 
holders by the end of 2011—12. Any further activities may 
require regulatory amendments and will depend on both the 
results of the above risk assessments and an analysis of existing 
legislation. 


Agreed. Health Canada will continue to assess whether 
resources have been allocated to identify risks by determining an 
appropriate risk-based inspection cycle and establishing 
appropriate baselines as described in the Department’s response 
to Recommendation 6.39. In addition, the Department will 
evaluate incident reporting through the Sentinel system and 
establish performance standards consistent with other product 
lines by the 2011-12 fiscal year. Data from medical device 
incident reports, other post-market activities, and associated 
performance measures will be used to identify high-risk areas 
requiring risk mitigation activities. The Department will also 
endeavour to determine, to the extent possible, whether the 
identified risks have been mitigated. 
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6.94 Health Canada should assess 
whether its communications to inform 
the public, industry, and health care 
professionals about risks associated with 
medical devices on the market are 
effectively reaching their target 
audiences and doing so in a timely 
manner. (6.92-6.93) 


6.98 Health Canada should determine 
whether it needs to take additional 
steps to increase international 
cooperation for its post-market 
activities. If so, the Department should 
identify concrete actions to be taken, 
along with timelines and needed 


resources. (6.95-6.97) 


Chapter 6 


Agreed. Health Canada will undertake a systematic evaluation 
of the effectiveness of its risk communications for all product 
lines, including medical devices. The plan will identify current 
gaps and challenges with risk communication evaluation and 
will highlight short-term, medium-term, and long-term activities 
aimed at systematic evaluation of Health Canada’s risk 
communication tools. This plan will be initiated in the 2011-12 
fiscal year with an anticipated completion date of the 2013-14 
fiscal year. 


Agreed. Over the next three years, Health Canada will identify 
gaps in international cooperation for post-market activities and 
the steps needed to increase international collaboration for post- 
market activities such as inspections, signal assessments, and 
communications. A three-year implementation work plan 


(2010-2013) will be developed. 
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Sheila Fraser, FCA 
Auditor General of Canada 


Progress made 


Foundations—Organizations that received 
significant amounts of federal funding in 
advance to carry out public policy objectives. 


Matters of Special Importance—2011 


Since I issued my first Report to the House of Commons in 2001, our 
audits have taken my Office into a wide range of government programs 
and activities. I continue to be impressed by the professionalism, 
commitment, and resilience of public servants and by their efforts to 
ensure that Canadians are well served by their government. I believe 
that, in their public service, Canadians have a world-class institution 
that should be a source of pride. 


My first Status Report to the House was tabled in September 2002. 
Our status reports tell parliamentarians what steps the government 
has taken to meet commitments made in response to our previous 
recommendations and whether, in our view, those steps represent 
satisfactory progress. In my view, there are numerous things that the 
federal government does well, particularly the ongoing activities on 
which many Canadians rely, including processing Canada Pension 
Plan, Old Age Security, and employment insurance benefits. 


In an organization as large and complex as our federal government, 
however, it is not surprising that we have also found areas that need 
attention. Many of the issues we audit are complex, and making progress 
takes sustained effort over time. A rating of satisfactory does not 
necessarily mean that no further action is needed, but rather that the 
action taken so far is reasonable over the period since our previous audit. 


I would like to use this Status Report—my last as Parliament’s Auditor 
General—to reflect on some examples of progress and also to underscore 
two challenges that remain of particular concern to me as my term 


comes to a close. 


One of the most satisfying aspects of my work has been the opportunity 
to point to good practices and real progress in a number of government 
programs. Taken together, our reports have included many examples of 
efforts by federal departments and agencies to make changes that will 
be of lasting benefit to Canadians. 


Actions taken toward improvement 


Foundations. The accountability of foundations is an example of 
progress arrived at in the past decade. In 1999 and for a number 

of successive years, the Auditor General’s observations on the 
government’s financial statements expressed concerns about the way 
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foundations were funded, the way their funding was accounted for, 
and their inadequate accountability for billions of dollars transferred 
to them by the government. 


Given the amount of money involved, I am very pleased that 
accountability to Parliament has been strengthened to the point where 
foundations now appear before parliamentary committees to discuss the 
management of their programs. The government has also strengthened 
its funding agreements with foundations to increase their 
accountability. 


I was pleased when Parliament amended the Auditor General Act, 
in June 2005, to give my Office a mandate to audit foundations. 
Where we have conducted audits of foundations—for example, 
Canada Health Infoway and the Canada Millennium Scholarship 
Foundation—we have generally reported positive observations. 


Passport Canada. Passport Canada had struggled in 2005 to meet 
service demands that were growing even as security requirements 
increased. In 2007, it faced a surge in applications for passports needed 
to fly to the United States, resulting in prolonged waits and delays for 
travelling Canadians. By 2009, we could report that Passport Canada 
had made satisfactory progress in addressing the problem and was 
preparing for any subsequent increase in passport applications leading 
up to June 2009, when Canadians would also need passports to enter 
the US by land or sea. 


The Canadian Food Inspection Agency. | have also noted the 
considerable efforts that the Canadian Food Inspection Agency has 
made since the 2004 avian influenza outbreak to improve its readiness 
for animal disease emergencies. The Agency developed policies, plans, 
and procedures to manage such emergencies and adjusted its approach 
based on lessons learned. While every animal disease emergency brings 
specific challenges that cannot always be anticipated, the Agency's 
established procedures enabled it to manage the 2007 and 2009 avian 
influenza emergencies and minimize the costs of lost production and 
the threats to animal health. 


Social insurance number (SIN). The management of the social 
insurance number is an area where we found limited progress in 2002. 
In 2007, we reported that, while Service Canada was heading in the 
right direction to improve data quality in the Social Insurance Register, 
overall progress was still unsatisfactory. I was pleased to note in our 
2009 Report, Managing Identity Information, that Service Canada 
had taken significant steps to implement a system for measuring and 


2 Status Report of the Auditor General of Canada—2011 


MATTERS OF SPECIAL IMPORTANCE—2011 


reporting on the quality of data in the Register. It also implemented a 
national quality control process to verify that new information being 
added to the Social Insurance Register at the time of a SIN application 
is complete, accurate, and valid. 


I hope that these few examples, selected from our reports over the 
years, will serve to assure Canadians that government does indeed take 
action to improve. I would like to turn now to an area of government 
where thi has been particularly evident—the government’s financial 
management. 


Significantly improved financial management practices 


I am pleased that I was able to provide a clean opinion on the 
government's summary financial statements, the Public Accounts of 
Canada, for each of my 10 years as Auditor General. A clean opinion, 
that is, without qualification, tells users of the financial statements that 
they can trust the figures presented and that they depict fairly the 
financial position. | congratulate the government on this 


accomplishment—an unqualified audit opinion is achieved by few 


governments in other countries. 


Another improvement I have noted during my term is the increased 
attention given by deputy heads to financial management. This may be 
due to the 2006 Federal Accountability Act, which designated deputy 
heads as the accounting officers of their departments. As such, they are 
accountable to the appropriate committee of Parliament to answer 
questions related to the management of their organizations. 


Another significant development was the creation of departmental 
audit committees, which advise the deputy heads. As we 
recommended in 2004, these committees are required to include 
experts from outside government, who have diverse professional 
backgrounds. These include academics, former deputy ministers, or 
distinguished individuals from the private sector. Departments have 
attracted Canadians with impressive credentials to serve on their 
departmental audit committees, and deputy heads have told us that 
the committees provide valuable, objective advice on the management 
of their organizations. 


As highlighted in Chapter | of this Status Report, another notable 
improvement in financial management is the recent requirement that 
departmental chief financial officers or their deputy chiefs have a 
recognized accounting designation. In 2003, when we first reported on 
the qualifications of senior financial officers, only 33 percent had 
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Remaining challenges 


accounting designations; in 2010, we found that 82 percent of chief 
financial officers in the 22 largest departments had professional 
accounting designations. This has increased the capability for effective 
management of financial matters in those departments. 


I am impressed by the progress shown in the short time since 2004 by 
internal audit, a key financial management tool discussed in Chapter 3 
of this Status Report. We said in 2004 that the success of internal audit 
would depend on its professionalism and the value it added to the 
department. A capable internal audit function provides senior 
management with objective, independent assurance that its financial, 
administrative, and operational controls and its management practices 
are effective. 


This year, I am pleased to report that there is greater appreciation of 
the role that internal audit can play; and, as a result, there is stronger 
senior management support for it across government. Departmental 
chief audit executives are now expected to have professional 
designations, and their audit staff are encouraged to seek their formal 
designations as certified internal auditors. 


The maturity and professionalism of the internal audit community was 
highlighted in our first phase of auditing the Economic Action Plan 
(reported in fall 2010). In that audit, we were able to rely on the work 
of internal audits completed in three departments and to incorporate 
the audit findings and conclusions into our own audit work. This 
coordination of work done by internal and external auditors enhanced 
the support that audit was able to provide to government operations. 


While progress in many areas has been noteworthy, the government 
faces some significant challenges. The following two, in particular, 
concern me: 


* Long-term fiscal pressures facing Canadians 


¢ Conditions on First Nations reserves 


Fiscal pressures 


Like governments in most countries, Canada’s federal government 
faces long-term fiscal pressures, such as those arising from the debt and 
deficit situation and an aging population. 


Under existing programs, an aging population will inevitably increase 
government spending on health care. Health care as a percentage of 
total program spending has risen in many jurisdictions. According to 
Statistics Canada, seniors aged 65 and older form the fastest-growing 
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segment of the population and will represent more than 25 percent 
of Canada’s population in 2056. 


As populations age, downward pressures will be brought to bear on 
labour force growth, economic growth, and tax revenues. The same 
people who will be needing more health care services, as they age, will 
also be drawing from the Canada Pension Plan, Old Age Security, and 
Guaranteed Income Supplement. Together, the growing cost of health 
care, increased retirement payments, and potentially lower tax 
revenues will present a major challenge. 


As noted by the Commissioner of the Environment and Sustainable 
Development, another looming fiscal pressure facing Canada is the 
cost of managing the potential impacts of climate change. Extreme 
weather events are increasing in number and severity; other impacts 
include threats to human health from extreme heat and the spread of 
diseases, such as the West Nile virus. The most extreme impacts are 
already occurring in Canada’s North where, for example, the thawing 
of permafrost is affecting the stability of roads, buildings, pipelines, and 
other infrastructure. Climate change will potentially have major 
impacts on key economic sectors as well, such as forestry, mining, 
fisheries, and agriculture. 


Adapting to climate impacts will be important to limit losses or to take 
advantage of opportunities that may accompany changing climate 
conditions. A national long-term strategy and action plan are needed 
to plan for adaptation, estimate the costs, and engage Canadians in 
adjusting their activities and their thinking. 


Several of our audits in the past decade have shown that the 
government will also face the challenge of replacing federal 
infrastructure—the Parliament buildings, bridges and ferries, mail 
handling facilities and equipment, research facilities and laboratory 
equipment, and the information technology systems that Canadians 
depend on for a variety of services and payments are but a few 
examples. Infrastructure replacement costs will add to the 
government’s long-term fiscal pressures. The cost of not developing 
and funding a long-term replacement plan will be far higher. 


I believe Canadians need sound, long-term fiscal information to 
understand the implications of policy choices, especially changes in 
spending, taxation, and debt levels. They need to know how these 
choices will affect the financial burden on present and future 
generations. Fiscal projections that look only a few years into the 
future will not give them what they need to meet the challenges ahead. 
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Nearly 30 countries and some Canadian provinces publish long-term 
” fiscal projections, ranging from 25 to 100 years into the future. For 

example, for the last 20 years, the United States has produced fiscal 

projections that extend 75 years. In Canada, the Department of 

Finance Canada last published long-term projections 9 years ago 

as informal working papers, which are long since outdated. 


I recognize the scale, complexity, and difficulty of managing these 
financial challenges. I encourage the government to publish the long- 
term financial projections needed to fully assess the impact of the 
challenges facing us and to inform Canadians and engage them in 
discussion about the difficult choices that need to be made. Success 
will require strong leadership and the ability to maintain focus and 
momentum over long periods of time. Resolving these long-standing 
challenges will take renewed vigour and clear plans and strategies. 

I encourage the government to engage Canadians in these decisions. 


Conditions on First Nations reserves 


Between 2001 and spring 2010, my reports included 16 chapters 
addressing First Nations and Inuit issues directly. Another 15 chapters 
dealt with issues of importance to Aboriginal people. I am profoundly 
disappointed to note in Chapter 4 of this Status Report that despite 
federal action in response to our recommendations over the years, 

a disproportionate number of First Nations people still lack the most 
basic services that other Canadians take for granted. 


It is clear that living conditions are poorer on First Nations reserves 
than elsewhere in Canada. In 2010, Indian and Northern Affairs 
Canada (INAC) reported that the average well-being of those 
communities continued to rank significantly below that of other 
Canadian communities. For example, the high school graduation rate 
for residents of reserves is 41 percent, compared with 77 percent for 
Canadians as a whole; and INAC data shows that more than half of 
the drinking water systems on reserves still pose a health risk to people 
who use them. In a country as rich as Canada, this disparity is 
unacceptable. 


On the surface, it may appear that the government simply needs to 
work harder to make existing programs work better. However, after 
10 years in this job, it has become clear to me that if First Nations 
communities on reserves are going to see meaningful progress in their 
well-being, a fundamental change is needed. 
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In my view, many of the problems faced by First Nations communities 
on reserves are due to structural impediments that severely limit the 
delivery of public services and hinder improvements in well-being. 

In Chapter 4 of this Status Report, we identify and provide details 

on four such impediments: a lack of clarity about service levels to 
First Nations, the lack of a legislative base, the lack of an appropriate 
funding mechanism, and a lack of organizations to support local 
service delivery. 


Real improvement will depend on the full participation of First Nations 
and the federal government; addressing the structural impediments 
will be a challenge for both parties. They will have to work together to 
address many obstacles. Unless they rise to this challenge, however, 
living conditions may continue to be poorer on First Nations reserves 
than elsewhere in Canada for generations to come. 


I believe that Canadians have a strong and highly capable public 
service. In ten years of serving as Auditor General, I have observed 
many things it does well—an important factor in why most Canadians 
enjoy an enviable standard of living. 


In terms of the future fiscal pressures we face as Canadians, I encourage 
the federal government to publish longer-term forecasts and to engage 
Canadians in discussions about choices that will have to be made. 


I also encourage the government and First Nations to recognize the 
need for a changed approach to services on First Nations reserves and 
to work together to improve the well-being of those communities. 


In closing, I thank my Office colleagues for their enthusiasm and 
commitment and their ability to deal with the many challenges we 
have faced together. I feel privileged to have worked with people 
who are so competent and so dedicated to serving Parliament and 
Canadians. I also acknowledge the excellent cooperation and 
assistance we received from the departments and agencies we audited, 
sometimes under trying circumstances. Finally, I want to express 
appreciation for the work of the committees of Parliament who 
reviewed our reports in committee hearings. With the continued 
combined efforts of these groups, I am confident that our democratic 
institutions will remain strong. 
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Main Points—Chapters 1 to 6 


Chapter 1 


What we examined 


Financial Management and Control 
and Risk Management 


Main Points 

Financial control and risk management are central to managing any 
organization effectively. To the extent that they are done well, they can 
contribute to an organization’s ability to safeguard its assets, use its 
resources economically and efficiently, and produce accurate and 
reliable financial information. Reliable information is key to 
developing a complete and accurate picture of financial performance. 


In our April 2003 Report we observed that federal departments we had 
selected for audit were in the early stages of implementing integrated 
risk management and that much remained to be done. For example, 
departments needed to complete their analysis of risk, including 
assessing their tolerance for risk. We also noted that departments 
needed more guidance from the Treasury Board of Canada Secretariat 
on the steps involved in integrated risk management. 


Our May 2006 Report noted that departments selected for that audit 
had made some progress in resolving financial control weaknesses and 
improving their financial management competencies. However, we 
noted that unless they also improved their financial controls, the 
quality of information used for many key decisions would be at risk. 
We also noted that most financial information used for budgeting and 
decision making was based on the cash method of accounting, which 
provided a less accurate and complete picture of a department's 
financial situation than the accrual method. 


In this follow-up audit we examined government’s progress in acting 
on commitments made in response to our previous concerns. Our 
audit included the Treasury Board of Canada Secretariat (the 
Secretariat) which includes the Office of the Comptroller General, and 
seven large federal departments, namely: Agriculture and Agri-Food 
Canada, Department of Finance Canada, Foreign Affairs and 
International Trade Canada, Human Resources and Skills 
Development Canada, Indian and Northern Affairs Canada, Transport 
Canada, and Veterans Affairs Canada. 


Audit work for this chapter was substantially completed 
on 30 September 2010. 
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Why it’s important 


Chapter 1 


What we found 


Canadians expect the government to be well managed and to be 
accountable for the safeguarding of public assets and the 
stewardship—efficient, effective, and economical use—of public funds. 
Effective financial controls and risk management. are vital to reduce 
the risk that the government may not achieve these objectives. 


Overall, the Treasury Board of Canada Secretariat and departments 
have made satisfactory progress in acting on their previous 
commitments, in particular, with respect to developing policies, 
frameworks, and relevant guidance on internal controls, and 
implementing risk management measures. Departments have still 
not fully assessed their internal control systems to identify and 
address weaknesses; and, according to their action plans, this work 
will take years to complete. In addition, the Office of the 
Comptroller General (OCG) has yet to establish processes for 
monitoring the completion of these assessments or for assessing 
actions taken by departments to address significant internal control 
issues. There is also a long-standing issue that has not been 
sufficiently addressed—namely, accrual-based budgeting and 
appropriations. 


The Treasury Board of Canada Secretariat, through the Office of the 
Comptroller General, has demonstrated leadership by developing a 
new and strengthened financial management policy suite, clarifying 
roles and responsibilities, and supporting deputy heads in their role 
as accounting officers as set out in the Financial Administration Act. 
Most of the selected departments told us they found the Secretariat’s 
initiatives and approach to be helpful. However, there is more work 
to be done to improve the Secretariat’s monitoring of the 
implementation of these policies so that it can appropriately assess 
departments’ performance in achieving the policy goals. 


Each of the selected departments has developed a corporate risk 
profile that summarizes an assessment of the department’s key risks, 
with processes in place to update these profiles regularly. The 
integration of risk into planning, reporting, and decision making has 
improved. 


Financial human resource capacity has improved since our last 
report. The Office of the Comptroller General has put in place 
several measures to enhance capacity, including guidelines on the 
qualifications of chief financial officers, mandatory training courses 
for financial staff, promotion of professional accounting designations 
for mid-level financial officers and financial managers, staffing 
initiatives, and forums and exchanges on best practices. 
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Departments are working to maintain and strengthen their 
workforce by recruiting, training, and developing the appropriate 
number of financial officers and managers with the required skills, 
competencies, and experience. Departments are at various stages in 
the process of putting in place strategies to address the anticipated 
turnover of senior financial executives. 


The government has not completed its evaluation of accrual-based 
budgeting and accrual-based appropriations and has not yet decided 
if it intends to fully adopt either or both of these approaches. It has 
taken what it considers the needed action related to accrual-based 
budgeting. However, it is our view that these actions are not 
sufficient or appropriate to attain accrual-based budgeting at the 
departmental level. At this point no decision on whether to adopt 
accrual-based appropriations has been made. 


The departments and the Secretariat have responded. The 
departments and the Secretariat agree with all of the recommendations. 
Their detailed responses follow the recommendations throughout 

the chapter. 
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Chapter 2 


What we examined 


Chapter 2 


Large Information Technology 
Projects 


Main Points 


Large information technology (IT) projects involve more than 
introducing new hardware and software and systems. These projects 
can introduce new processes and practices—new ways of doing 
business—that also need to be successfully implemented before 
organizations can take advantage of potential efficiencies and savings. 
When successful, these projects can change the way that departments 
carry out their work and improve services to Canadians. Because large 
IT projects are complex and costly, they usually involve long planning 
and development times and require significant investments (on 
average, over three years and more than $70 million). 


In 2006, we looked at seven large IT projects and found that only 
two of them met all of our audit criteria for well-managed projects. 
We found that five of the seven projects had proceeded even though 
their business cases were incomplete or out of date or contained 
information that could not be supported. In addition, four of the 
projects were undertaken by departments that lacked the required 
skills and experience to manage them. Although by 2006 the Treasury 
Board of Canada Secretariat had established a framework of best 
practices for managing IT projects, many of the problems identified 
some nine years earlier had persisted. In 2006, we also found that 
limited progress had been made since our previous audit in 1997. 


For this status report, we examined the progress made since 2006 
by the four departments that had not met all of our criteria in 2006. 
We also selected a new project, approved by the Treasury Board 
since 2006, in order to assess the government’s progress in the way 
it approves and manages large IT projects. 


In our 2006 audit, we were denied access to information by the 
Secretariat, which prevented us from completing our review of its 
challenge role in support of oversight of large IT projects by Treasury 
Board ministers. In this audit, we were able to review information that 
demonstrated the role played by the Secretariat. 


Audit work for this chapter was substantially completed 
on 29 October 2010. 
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MAIN PoINTS—CHAPTERS 1 To 6 


The federal government relies on information technology systems to 
provide many programs and services to Canadians. Large IT projects 
are inherently complex, expensive, and risky. Since 2002, the federal 
government has approved funding of $7.5 billion for new business 
projects making significant use of information technology. In the 
four audits we performed since 1995, the projects audited have a 
history of cost overruns and delays, and of not delivering what had 
been planned originally. 


Overall, the government has made unsatisfactory progress on 

its commitments in response to our 2006 recommendations. Although 
some improvements have been made, progress has been unsatisfactory 
in the important areas of governance and project business cases. 
Only two of the five projects we looked at, the Temporary Resident 
Biometrics Project and Integrated Revenue Collections systems, met 
most of our criteria for well-managed projects. 


In order to increase the likelihood of success, departments have 
significantly reduced the scope of the projects and considerably 
extended their timelines. In the area of project governance, the 
Expenditure Management Information System and Global Case 
Management System have had important deliverables deferred 
without full analysis of the impacts and costs of not completing these 
projects. In three of the five projects examined, the project business 
cases did not identify measurable benefits or the benefits have not 
been measured. For example, the 2007 business case for the Secure 
Channel no longer quantifies the financial benefits. 


In three of the four projects we examined, departments adequately 
assessed their capacity to manage the projects and demonstrated that 
they were ready to accept the business transformation that came 
with them; in three of the four projects, departments have done an 
adequate job of managing projects risks. 


The Secretariat has met the commitments it made in response to our 
recommendations we addressed to it. Since 2007, it has completed a 
policy suite renewal, which has led to new policies, standards, and 
guidance that will directly impact information technology projects by 
the end of the five-year implementation period. The Secretariat has 
been actively challenging departments in their preparation of their 
IT project submissions; recently (September 2010) it also submitted 
its first semi-annual report to Treasury Board ministers summarizing 
the health of large IT projects across government. That report 
provided a snapshot of 12 ongoing projects with a total value of 
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$2.4 billion. However, it is too early to assess whether these measures 
will have a significant impact on the management of large IT projects 
by departments. 


The organizations have responded. The organizations agree with all 
of the recommendations. Their detailed responses follow the 
recommendations throughout the report. 
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Chapter 3 


What we examined 


Why it’s important 


Internal Audit 


Main Points 


In November 2004, we reported considerable variation among 

six federal organizations in the extent to which their internal audit 
activity met the international standards for the professional practice of 
internal auditing issued by the Institute of Internal Auditors. We also 
reported the extent to which they complied with the Treasury Board 
Policy on Internal Audit. We found that the Treasury Board of Canada 
Secretariat had yet to establish and fund a strategy that would enable it 
to meet the requirements of the Policy and the expectations of the 
internal audit community. We made a number of recommendations 
aimed at improving the quality of internal audit across government. 


Our 2004 report noted that, effective 1 June 2004, the government 
had re-established the Office of the Comptroller General to strengthen 
comptrollership and oversight across the federal government. The 
Comptroller General’s key duties included setting or reviewing internal 
auditing standards and policies of the Government of Canada, 
providing leadership to ensure and enforce appropriate internal 
controls, and promoting sound resource stewardship at all levels across 
the federal government. 


For this status report, we examined the extent of progress made by the 
government in acting upon the commitments it made in response to 
the observations and recommendations of our 2004 report. We looked 
at whether independent audit committees had been established in the 
24 largest departments and agencies that represent about 95 percent of 
the government's total assets, liabilities, revenues, and expenses. We 
assessed the practices and procedures used by a sample of internal 
audit activities. We also looked at whether the Office of the 
Comptroller General provided appropriate oversight and guidance to 
the internal audit activity in departments. 


Audit work for this chapter was substantially completed on 


28 November 2010. 


Internal audit is an important element of good governance. An 
effective internal audit activity can provide senior management with 
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objective, independent assurance that the organization’s financial, 
administrative, and operational controls and management practices 
are effective. It can also suggest improvements that may be needed. 


The government has made satisfactory progress in acting upon the 
commitments it made in response to the observations and 
recommendations of our 2004 audit. 


Independent departmental audit committees have been established 
in the 24 large departments; the majority of their members are from 
outside the federal public administration and have the collective 
skills and experience required to provide deputy heads of 
departments with objective advice and recommendations. 
Departmental audit committees are at varying stages in developing 
their practices and procedures. Some committees have been in effect 
for three years and others less than one year. We noted that the 
Canada Revenue Agency established its audit committee in 1999. 
We have also noted the positive impact that established 
departmental audit committees are having in contributing to 
stronger internal audit in government. 


The government has strengthened its internal audit capacity since 
our last audit in 2004. While few internal audit activities in 
departments have undergone an external quality assessment review, 
our own review of a sample of internal audit activities found that 
they generally or partially conformed to policy requirements and to 
the standards of the Institute of Internal Auditors. Internal audit 
reports we reviewed met those standards and requirements. We 
noted that the strong senior management support for internal audit, 
coupled with the impact of departmental audit committees has been 
accompanied by a strengthened internal audit capacity. Senior 
management has indicated that it has a greater appreciation of the 
role that internal audit can and should play within an organization. 


The Office of the Comptroller General has provided direction and 
guidance to the internal audit and departmental audit committee . 
communities through the establishment of a sound policy framework 
and related guidance. We noted, for example, orientation material 
that included roles, responsibilities, and expectations of audit 
committee members. The Office of the Comptroller General has also 
developed tools and guidance for the internal audit community, such 
as risk-based audit plans and guidance on core controls. 


The entities have responded. The entities agree with our 


recommendation. Their detailed responses follow the recommendation 
in the chapter. 


Status Report of the Auditor General of Canada—2011 


Chapter 4 


What we examined 


Why it’s important 


Programs for First Nations 
on Reserves 


Main Points 


The federal government supports numerous services to First Nations 
members on reserves that are similar to those provided by provincial 
and municipal governments to people off reserves. These services 
range from education and social development programs to land 
management and economic development, housing, provision of 
potable water, and provision of benefits to First Nations members 
under treaties and other agreements. 


In several audits over the past decade, we have identified issues of 
particular importance to the lives and well-being of First Nations and 
Inuit. In this follow-up audit, we examined the government’s progress 
toward achieving the commitments it made to address significant 
observations and recommendations from seven of those reports, issued 
between 2002 and 2008. We focused on the areas of education, water 
quality, housing, child and family services, land claim agreements, and 
reporting requirements. 


Our audit included the roles that Indian and Northern Affairs Canada 
(INAC), Health Canada, the Canada Mortgage and Housing 
Corporation (CMHC), and the Treasury Board of Canada Secretariat 
play in the management of programs to improve the lives of First 
Nations and Inuit. 


Audit work for this chapter was substantially completed 
on 1 November 2010. 


According to the Report of the Royal Commission on Aboriginal 
Peoples (1996), Canada’s First Nations reserves experience long- 
standing challenges such as ill health, insufficient and unsafe housing, 
high unemployment, polluted water supplies, inadequate education, 
poverty, and family breakdown. The federal government supports 
programs that can improve the lives of people who live on reserves. 
However, despite the billions of federal dollars spent each year on 
programs to address the social and economic challenges on First 
Nations reserves, conditions there remain significantly below the 
national average. 
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Chapter 4 


What we found 


Despite considerable efforts and concrete actions in some areas, 
federal organizations have not made satisfactory progress on 
commitments made in response to several recommendations from 
our previous audits. INAC has just begun implementing a strategy to 
close the education gap. The Department has not defined what is 
meant by its policy of providing child and family services that are 
reasonably comparable to what exists in the provinces. Nor has it 
developed and implemented a plan to communicate to other federal 
organizations what specific obligations they have under land claim 
agreements or monitored their results. INAC and Health Canada do 
not ensure that drinking water is tested on a regular basis. INAC, 
Health Canada, and the Canada Mortgage and Housing Corporation 
have developed a strategy for dealing with mould in housing, but it is 
not comprehensive as it focuses on education rather than 
remediation and provides no new funding. 


INAC has implemented commitments it made in response to some of 
our recommendations. For example, it has taken steps to monitor 
progress toward the objectives of comprehensive land claim 
agreements, and has put in place a system to track and monitor 
progress on all federal commitments contained in the agreements. 
Along with Health Canada, INAC has developed draft legislation 
related to drinking water on reserves. In addition, the Department 
recently agreed with several provinces and First Nations on 
frameworks to improve child and family services on reserves, but it 
will likely be several years before meaningful results are realized. 


Notwithstanding the considerable efforts made, conditions have 
generally not improved for First Nations in each of the areas 

subject to our audit. The education gap between First Nations 
living on reserves and the general Canadian population has widened, 
the shortage of adequate housing on reserves has increased, 
comparability of child and family services is not ensured, and the 
reporting requirements on First Nations remain burdensome. 


Broader concerns that we believe have inhibited progress include 
the lack of clarity about service levels on First Nations reserves, lack 
of a legislative base to fund service delivery on reserves, a lack of an 
appropriate funding mechanism, and a lack of organizations that 
could support local service delivery. There is a risk that living 
conditions on many First Nations reserves will remain significantly 
below national averages, with little prospect of a brighter future, 
until these concerns are addressed. 


The Department has responded. The Department agrees with our 
recommendation. Its detailed response follows the recommendation. 
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What we examined 


Why it’s important 


National Police Services— 
Royal Canadian Mounted Police 


Main Points 


The Royal Canadian Mounted Police (RCMP) provides Canada’s law 
enforcement community with specialized national police services such 
as forensic analyses of criminal evidence, criminal records information, 
identification services, technological support, learning opportunities, 
and the collection and analysis of criminal information and 
intelligence. 


This follow-up audit assessed the progress that the federal government 
and the RCMP have made toward keeping commitments to improve 
certain national police services, including their timeliness, made in 
response to our audits in 2000, 2004, and 2007; those audits noted 
significant delays in the delivery of these services. We focused on 

four of the national police services: Forensic Laboratory Services, the 
National DNA Data Bank, Criminal Intelligence Service Canada, and 
Canadian Criminal Real Time Identification Services. The DNA Data 
Bank has not been the subject of a previous audit by this office. 


We also followed up on previous audit observations concerning the 
governance of national police services as a whole and the approach to 
funding them. 


Audit work for this chapter was substantially completed 
on 1 November 2010. 


Jurisdictional limits on individual police forces, the division of 
investigative responsibility when crime crosses these jurisdictional 
lines, and constraints on funds and manpower limit the ability of any 
single police force to carry out complex investigations. Given the 
RCMP’s federal and contractual policing responsibilities, provincial 
and municipal police forces often rely on the RCMP to provide highly 
specialized police support services. The aim of providing national 
police services is to enable and sustain uniform access to information 
that supports public safety and the administration of justice to all 
Canadians, particularly where such services may not be within the 
resources of individual jurisdictions. 
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Chapter 5 


What we found 


- The federal government and the Royal Canadian Mounted Police 
have made unsatisfactory progress on commitments made in 
response to recommendations in our 2000, 2004, and 2007 reports 
regarding national police services. The RCMP is still providing 
national police services to other levels of government and law 
enforcement organizations without clarifying which services should 
be provided; how they should be funded; and how provincial, 
territorial, and municipal partners should be engaged within the 
national police services governance framework. In the absence of 
formal agreements, the accountability of the RCMP and other 
partners for the delivery of these services has not been clearly 
established, nor has how they are to be sustained over the long term. 


* The RCMP has not determined the detailed transactional costs of 
providing each of its national police services. Accurate costing 
information is essential to efficiently manage services and resources. 
In addition, because of financial pressures, the RCMP is finding it 
difficult to deliver national police services and also fulfill its other 
obligations. In light of existing resource constraints, it is difficult 
without adequate cost information to negotiate agreements on 
which national police services will be provided and which are 
priorities. 


While changes in the approach to managing Forensic Laboratory 
Services have increased the RCMP’s focus on clients’ needs and 
reduced turnaround times, other services have not shown similar 
improvement. For example, backlogs and delays in updating criminal 
record information have significantly increased—from 5 months 

in 2000 to 14 months today. In addition, technological upgrades to 
critical systems requiring significant investments are facing 
challenges or have been delayed. At the same time, the demand for 
national police services is growing and straining the RCMP’s ability 
to deliver in a timely way. 


The RCMP and Public Safety Canada have responded. The RCMP 
and Public Safety Canada agree with all of the recommendations. 
Their detailed responses follow the recommendations throughout the 
chapter. 
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What we examined 


Why it’s important 


Regulating Medical Devices— 
Health Canada 


Main Points 


‘ 


The term “medical devices” encompasses a wide range of medical, 
surgical, and dental products and instruments used to diagnose, treat, 
and prevent diseases and other physical ailments. They range from 
basic items, such as bandages, to complex devices such as pacemakers 
and diagnostic ultrasound systems. 


As required under the Food and Drugs Act, Health Canada regulates 
the safety and effectiveness of all medical devices marketed in Canada. 
It does this through a combination of scientific review, monitoring, 
compliance, and enforcement activities, both before and after the 
devices reach the Canadian marketplace. The Department works 
toward ensuring that the public has timely access to safe and effective 
medical devices, and that those who need to know about related safety 
concerns are informed of them in a timely manner. 


In March 2004 we reported concerns about the continued viability of 
the Medical Devices Program, given the increasing complexity and 
number of devices available in Canada and the resources allocated to 
the Program. We concluded that the Program was not sustainable with 
the resources it had. We made similar observations in our 


November 2006 Report. 


In this audit, we examined Health Canada’s progress in meeting 
commitments it made in response to selected recommendations from 
our previous reports. In particular, we examined the timeliness of the 
Department’s review of applications to allow or deny medical devices 
to be sold in Canada. We also examined how it manages the risks 
related to medical devices already available on the Canadian market 
and how it meets its obligations for medical devices. In addition, we 
looked at international regulatory cooperation. 


Audit work for this chapter was substantially completed on 
29 October 2010. 


Medical devices play an important role in all stages of the delivery 
of quality health care, from the most basic procedures, such as 
monitoring blood pressure and body temperature, to complex surgeries. 
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Chapter 6 


What we found 


All are meant to improve the health and well-being of patients by 
helping to prevent, diagnose, and treat disease, to reduce pain and 
suffering, and to extend and save lives. Medical devices are increasing 
in number and complexity, due in part to medical and technological 
advances. 


Close to 1.4 million different medical devices are currently on the 
Canadian market. Canadians rely on Health Canada for assurance that 
available medical devices are safe and effective. Users of medical 
devices also rely on Health Canada to promptly inform them of safety 
concerns related to medical devices that are in use. 


Overall, Health Canada has made unsatisfactory progress toward 
meeting commitments it made in response to recommendations in 
our 2004 and 2006 reports. Although it has accomplished much in 
some areas, the Department is not meeting its obligations under the 
Medical Devices Program, mostly at the pre-market stage. For 
example, more than 45 percent of the time it does not meet its 
service standards for timely review of medical device submissions, 
thus delaying Canadians’ access to the health benefits of these 
devices. According to Health Canada, this problem is due to a 
funding shortfall caused in part by rapid growth in medical device 
technology and increasingly complex submissions. 


Health Canada does not know if it has allocated too much or not 
enough resources to post-market activities such as inspections and 
surveillance. The Department has not established what levels of 
activity are needed to protect the health and safety of Canadians. In 
addition, while the Department has identified risks associated with 
medical devices already available on the Canadian market, it has yet 
to determine whether the risks that the inspections and the review of 
incident reports are designed to address have been adequately 
mitigated. 


Health Canada recently developed strategies to increase cooperation 
at the pre-market stage with regulatory bodies in other countries. 
While we acknowledge the significant challenges involved, progress 
to date is unsatisfactory. The work has not taken advantage of 
foreign resources, information, and knowledge and has not yet 
improved regulatory performance. 


Health Canada has taken significant actions to meet some 
commitments made in response to other recommendations from 
previous reports. A major accomplishment was making the Medical 
Devices Program more sustainable than in previous years. The 
Department increased its funding to the Program and expects to 
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eliminate the remaining funding shortfall by implementing new 

user fees starting in the 2011-12 fiscal year. Another significant 
accomplishment has been establishing a national inspection program 
to assess compliance with the Food and Drugs Act and Regulations. 
The Department also increased its capacity to identify safety risks 
associated with medical devices on the market. 


The Department has responded. The Department agrees with all of 
the recommendations. Its detailed responses follow the 
recommendations throughout the chapter. 
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APPENDIX 


Appendix Report on the audit of the President of the Treasury Board's report, Zab/ing of Grown 
Corporations’ Reports in Parliament 


Tablings in Parliament for parent Crown corporations: Annual reports and summaries of corporate plans and budgets 


Section 152 of the Financial Administration Act (the Act) requires the President of the Treasury Board to 
lay before each House of Parliament, no later than 31 December of each year, a report on the timing of 
the tabling, by appropriate ministers, of annual reports and summaries of corporate plans and of budgets of 
Crown corporations. The Act also requires the Auditor General of Canada to audit the accuracy of the 
report on the timing of tabling and to present the results in her annual report to the House of Commons. 


The President of the Treasury Board’s report on the timing of tabling is included in the Annual Report 
to Parliament—Crown Corporations and Other Corporate Interests of Canada 2010, which was tabled 
on 8 December 2010. 


At the time that our October 2010 report was going to print, we were unable to include the results of 
the above-noted audit because the President of the Treasury Board’s report had not yet been finalized. 
Our Auditor’s report was subsequently appended to the President’s report and is reproduced in this report 
to Parliament. 
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Auditors’ Report 
To the House of Commons: 


As required by subsection 152(2) of the Financial Administration Act, I have audited, for the year 

ended 31 July 2010, the information contained in the report “Tabling of Crown Corporations’ Reports in 
Parliament” included in the Annual Report to Parliament—Crown Corporations and Other Corporate Interests 
of Canada 2010. The information contained in the report is the responsibility of the President of the 
Treasury Board. My responsibility is to express an opinion on the information contained in the report 
based on my audit. 


I conducted my audit in accordance with the standards for assurance engagements established by 

The Canadian Institute of Chartered Accountants. Those standards require that I plan and perform 

an audit to obtain reasonable assurance as to whether the information contained in the report is free of 
significant misstatement. My audit included examining, on a test basis, evidence supporting the dates 
and other disclosures provided in the report. 


In my opinion, the information contained in the report “Tabling of Crown Corporations’ Reports in 
Parliament” is accurate, in all significant respects, with the section “Deadlines for tabling in Parliament 
and results achieved” contained within the report. 


Lucie Cardinal, CA 
Principal 
for the Auditor General of Canada 


Ottawa, Canada 
15 November 2010 
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Special Examinations of Crown Corporations 


711 This chapter presents the main points of special examination 
reports of Crown corporations that were issued to the corporations’ 
boards of directors between | January 2010 and 31 December 2010 
and that have subsequently been made public. The Office of the 
Auditor General transmitted these reports throughout the year and 
examined the respective systems and practices during different 
time periods. 


7.2. Crown corporations are distinct legal entities whose names, 
mandates, powers, and objectives are set out in either the constituent 
legislation for the parent Crown corporation or the articles of 
incorporation under the Canada Business Corporations Act. 


7.3 Crown corporations have more autonomy than most other 
government entities, in part, because they have commercial objectives 
as well as public policy objectives. They account for a significant 
portion of government activity and operate in many sectors of the 
Canadian economy, including transportation, energy, agriculture and 
fisheries, financial services, culture, and government services. 


74 As of 31 December 2010, 46 parent Crown corporations and 

3 subsidiaries reported as parent Crown corporations. These Crown 
corporations employed over 91,000 people. More than 81 percent of 
these employees worked at four corporations: Canada Post 
Corporation, Canadian Broadcasting Corporation, Atomic Energy of 
Canada Limited, and VIA Rail Canada Inc. The vast majority of 
Crown corporations had fewer than 3,000 employees. 


75 Crown corporations manage more than $350 billion in assets. 
Note that due to the unique nature of their operations, total asset 
holdings for the Bank of Canada, the Canada Pension Plan Investment 
Board, and the Public Sector Pension Investment Board have not been 
included in the Crown corporations’ total assets. Crown corporations 
fund their operations in a variety of ways. Some corporations are 
required by their enabling legislation to be financially self-sustaining 
and thus receive no parliamentary appropriations. Some are funded 
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Role of the Office of 
the Auditor General 


Definition and objective of 


a special examination 


Significant deficiency—A major weakness ina 
Crown corporation's key systems and practices 
that could prevent it from having reasonable 
assurance that its assets are safeguarded and 
controlled, its resources are managed efficiently 
and economically, or its operations are carried 


out effectively. 


Chapter 7 


mainly through parliamentary appropriations. Others receive federal 
funding but also generate revenue. In the 2009-10 fiscal year, 

26 corporations received just over $6.6 billion in appropriations; 

82 percent of these appropriations went to 5 corporations: Canada 
Mortgage and Housing Corporation, Canadian Broadcasting 
Corporation, Atomic Energy of Canada Limited, Canadian Air 
Transport Security Authority, and VIA Rail Canada Inc. 


76 | New Crown corporations are created periodically. The Canadian 
Museum of Immigration at Pier 21 was the only Crown corporation 
created in 2010. The Government dissolved the Cape Breton 
Development Corporation and transferred its remaining assets and 
liabilities to the Enterprise Cape Breton Corporation. 


711. A board of directors or a similar governing body oversees the 
management of each corporation and holds management responsible 
for the corporation’s performance. The board of a parent Crown 
corporation is, in turn, accountable to Parliament through the 
responsible minister. 


78 Under Part X of the Financial Administration Act, the Auditor 
General is appointed to conduct annual audits of the financial 
statements and periodic special examinations of Crown corporations, 
either on her own or jointly with a private sector audit firm, unless she 
waives the appointment. 


79 Aspecial examination is an important accountability mechanism 
for Crown corporations. The objective of a special examination is to 
provide an independent opinion on whether there is reasonable 
assurance that a Crown corporation has systems and practices in place 
to ensure that its 


* assets are safeguarded and controlled; 


- financial, human, and physical resources are managed economically 
and efficiently; and 


* operations are carried out effectively. 


7.10 Any major weakness in the key corporate systems and practices 
that could prevent a corporation from achieving these objectives is 
reported as a significant deficiency. When planning special 
examinations, we conduct a risk analysis to identify the systems and 
practices that we consider essential to providing the corporation with 
this assurance. We also establish criteria that we use to examine the 
corporation’s systems and practices. These criteria are based on our 
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experience with performance auditing and our knowledge of the 
subject matter. They are selected in consultation with the corporation. 


711 The opinion we present in the special examination can take one 
of three forms. One conclusion is that there is reasonable assurance 
that there are no significant deficiencies in the systems and practices 
the corporation maintained to achieve the objectives noted above. 
The second is that there is one or more significant deficiencies. Lastly, 
in rare situations, we may find there is not reasonable assurance that a 
corporation’s systems and practices achieve the objectives. 


7.12 The frequency of a special examination, as determined by the 
Financial Administration Act, is at least once every 10 years. However, 
special examinations could be carried out more frequently, if required 
by the Governor in Council (the Cabinet acting in a legal capacity), 
the appropriate minister, the board of directors of the corporation, or 
the Auditor General. 


7.13 As part of the accountability mechanism for Crown corporations, 
the Office reports on special examinations to the board of directors of a 
Crown corporation or any similar governing body. Furthermore, the 
Auditor General may bring information from the special examination 
to the attention of the appropriate minister, after consulting with the 
board of directors or any similar governing body. The Auditor General 
may also bring such information to Parliament’s attention, after 
consulting with the appropriate minister and the board of directors, 
by preparing a document to be included in the corporation’s annual 
report. 


714 The Financial Administration Act requires that the board of 
directors submit all special examination reports to the appropriate 
minister and the President of the Treasury Board within 30 days of 
receiving them. The Act also requires that the board of directors make 
the reports available to the public within 60 days of receiving them. 


7.15 Reports were issued for the following four Crown corporations 
between | January 2010 and 31 December 2010, and have been 
subsequently made public: 


* Canada Deposit Insurance Corporation 
« Freshwater Fish Marketing Corporation 
« National Arts Centre Corporation 


- Telefilm Canada 
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716 Of these four special examination reports, two identified no 
significant deficiencies. Two identified one or more significant 
deficiencies: 


+ The National Arts Centre Corporation had a significant deficiency 
in the Corporation's systems and practices for the maintenance of 
the centre. 


The Freshwater Fish Marketing Corporation had significant 
deficiencies in the Corporation’s systems and practices related to 
governance, strategic planning, and risk management. It also had 
significant deficiencies in operational areas, including processes to 
establish prices paid to fishers in order to remain self-sufficient and 
meet its mandate, capital asset management and procurement, 
production efficiency, and human resources. Due to these significant 
deficiencies, we concluded that the Freshwater Fish Marketing 
Corporation did not have the reasonable assurance, as required by 
section 131 of the Financial Administration Act, that its assets were 
safeguarded and controlled, its resources were managed economically 
and efficiently, and its operations were carried out effectively. 


We chose to bring the results of the Freshwater Fish Marketing 
Corporation examination to the attention of the appropriate minister. 


717 In addition to reporting on significant deficiencies, our special 
examinations highlight systems and practices that contribute to 
success, and they provide information and recommendations to boards 
of directors about other opportunities for improvement. For the 

four Crown corporations, our recommendations focused primarily on 
corporate governance, strategic planning, risk management, and 
performance measurement and reporting. Because of the small number 
of special examination reports that we issued during this period, it is 
difficult to identify overall trends. 


7.18 As noted earlier, we transmitted these reports to the respective 
board of directors at different dates throughout the year and examined 
the respective systems and practices during different time periods. We 
have not performed any follow-up audit work regarding any of the 
matters raised in these reports. 


719 The next section includes the main points from each of the 
four special examination reports and not the full reports. Full reports 
can be found on the respective corporation’s website. 
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Main Points of Special Examinations—2010 


Canada Deposit Insurance Corporation—Main Points 


[We issued the full report to the board of directors on 4 March 2010. 
The examination work covered the period from February 2009 to July 2009. 
For the full report, please go to www.cdic.ca or contact the Corporation. ] 


Canada Deposit Insurance Corporation (CDIC) is a federal Crown 
corporation with a mandate to insure savings deposited in a CDIC 
member institution against loss in the event that the institution fails. 
The Corporation is also to promote and otherwise contribute to the 
stability of the financial system in Canada. The Corporation reports to 
Parliament through the Minister of Finance. . 


CDIC was established in 1967 under the Canada Deposit Insurance 
Corporation Act. Its head office is located in Ottawa, and there is one 
office in Toronto. At 31 March 2009, the Corporation had 


97 permanent employees. 


We examined whether CDIC’s systems and practices provide it with 
reasonable assurance that its assets are safeguarded and controlled, its 
resources are managed economically and efficiently, and its operations 
are carried out effectively. Our examination covered the systems and 
practices in place between February 2009 and June 2009, including 
those in the areas of corporate governance, strategic direction and risk 
management, intervention and risk assessment, and human resources. 


The federal oversight and safety net system for deposit-taking 
institutions are essential to the stability of the Canadian financial 
system. The overwhelming majority of Canadians and Canadian 
businesses have deposits that are insured by CDIC, whose primary role 
is to protect these depositors—in the event of a member institution 
failure. Canadians expect CDIC to be able to fulfill this role. 


We found no significant deficiencies in the Corporation’s systems and 
practices. In the areas where we focused our examination, we noted a 
number of good systems and practices. Some examples are outlined 
below. 


* CDIC has the key elements of a good governance framework and 
its governance practices are consistent with best-practice 
expectations for stewardship by boards of directors. In addition, its 
systems and practices for human resources management provide 
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the Corporation with the core competencies and skills it needs to 
meet its objectives. 


The Corporation identifies and monitors risk and reports on its 
risk assessment to the Board’s Audit Committee each year. It also 
carries out ongoing risk assessment of its member institutions, 
which enables it to keep track of developments affecting them and 
the financial services sector in general. Since our previous special 
examination in 2004, it has improved its readiness to intervene in 
the affairs of a troubled member. 


CDIC has a sound strategic planning process. It could, though, 
improve its reporting on performance by including in its corporate 
scorecard not only the planned key initiatives and key 
performance indicators currently presented but also an indication 
of its progress toward meeting its stated intended outcomes. 


The Corporation has responded. The Corporation agrees with our 
recommendation. 
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Freshwater Fish Marketing Corporation—Main Points 


[We issued the full report to the board of directors on 15 December 2010. 
The examination work covered the period from January 2009 to 

March 2010. For the full report, please go to www.freshwaterfish.com 

or contact the Corporation. ] 


The Freshwater Fish Marketing Corporation was established in 1969 to 
market and trade in Canadian freshwater fish and fish products and by- 
products. The Corporation reports to Parliament through the Minister 
of Fisheries and Oceans. It is required to be financially self-sustaining 
and operates without government funding, although the Government 
of Canada guarantees the Corporation’s borrowing up to a legislative 
limit of $50 million. 


The Corporation has 53 full-time employees in corporate functions 
such as marketing, finance, and human resources. It also has the 
equivalent of approximately 150 full-time employees working in 
production; while the number of employees can increase during peak 
production periods. The Corporation buys around 17 million kilograms 
of fish each year at more than 50 collection points on over 265 lakes; 
the fish are then processed at the Corporation’s plant in Winnipeg. 
From 2006 to 2010, the Corporation’s total annual sales were between 
$55 million and $64 million each year. Over seventy percent of these 
sales are to markets outside Canada. 


We examined whether the Freshwater Fish Marketing Corporation’s 
systems and practices provide it with reasonable assurance that its 
assets are safeguarded and controlled, its resources are managed 
economically and efficiently, and its operations are carried out 
effectively. We focused on the following areas: corporate governance; 
strategic planning and risk management, and prices paid to fishers; 
performance management; production efficiency; quality 
management; human resource management; capital asset management 
and procurement; marketing and sales planning; and inventory 
management. Our examination covered the systems and practices that 
were in place between January 2009 and March 2010. 


The fish in Canada’s many lakes and rivers are a very valuable resource 
that the Corporation promotes and brings to market around the world. 
Working with fishers through agreements with Alberta, Saskatchewan, 
Manitoba, Ontario, and the Northwest Territories, the Corporation has a 
mandate to purchase fish that are legally caught and sold, create an 
orderly market, promote freshwater fish in international markets, increase 
the size of the fish trade, and increase the profits earned by fishers. 
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The Corporation currently faces several critical challenges to its 
continued ability to be financially self-sustaining, including its rising 
debt levels, aging physical assets, the announced intent of Northwest 
Territories and Saskatchewan to withdraw from the agreement each 
signed with the federal government, and foreign currency fluctuations. 
These challenges will need to be well managed if the Corporation is to 
successfully fulfill its mandate and objectives. 


Our examination found significant deficiencies in the Corporation’s 
systems and practices related to governance, strategic planning and risk 
management; and in operational areas, including processes to establish 
prices paid to fishers in order to remain self sufficient and meet its 
mandate, capital asset management and procurement, production 
efficiency, and human resources. A significant deficiency is a major 
weakness that could prevent the Corporation from having reasonable 
assurance that its assets are safeguarded and controlled, its resources 
are managed economically and efficiently, and its operations are carried 
out effectively. The deficiencies we found are explained below. 


* While we understand that the Corporation has focused on 
keeping its plant open by addressing the most serious issues raised 
by regulators and others, it has not successfully addressed its key 
strategic challenges. The Corporation has not been able to obtain 
approval from the Government of Canada (the Corporation’s 
shareholder) of a viable plan that would enable the Corporation 
to continue to operate as a financially self-sustaining entity and 
fulfill its mandate. The Corporation has significant shortcomings 
in its governance practices: The Board is not focused on high risk 
issues; the current Board profile does not reflect the specific skills 
and experience required; the Audit Committee is not in 
compliance with the Financial Administration Act; the Corporation 
does not have internal audits conducted as required by the Act; 
and potential conflicts of interest on the part of Board members 
are not well managed. 


* In addition to lacking a corporate plan that would be approved by 
the government and would address the financial issues, the 
Corporation has not put in place a risk management process to 
provide reasonable assurance that it will achieve its strategic and 
operational objectives, manage its resources economically and 
efficiently, and protect its assets. The Corporation does not have a 
current strategic plan to overcome its critical challenges. The last 
strategic plan was approved by the Board of Directors in 2007; 
since then, much has changed in the Corporation’s internal and 
external operating environment. 
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* The Corporation has not met its annually budgeted net income 
levels over the past five years and is struggling financially. It has 
significantly increased its debt levels in recent years to finance 
capital additions and meet working capital requirements, and is 
now approaching its approved borrowing limits. The 
Corporation's practices for setting prices paid to fishers have not 
provided for the forecasted amounts of capital reinvestment 
needed. Consequently the Corporation has not reinvested the 
necessary capital in its aging production facility to safeguard its 
assets. Without the reinvestment there is a significant risk that 
the Corporation will not be able to operate efficiently and 
continue to fulfill its mandate..In addition, the Corporation’s 
systems and practices do not ensure that capital assets are 
procured in an economical and efficient manner. For example, 
there are insufficient contracting policies for large capital 
contracts and no established approach to ensure that all 
significant procurement risks are assessed and managed. 


* In our 2005 Special Examination Report, we considered yield 
measurement practices to be a serious shortcoming. Five years 
later no substantive change has been made to adequately address 
this issue. Currently the Corporation does not have an effective 
method of monitoring yield and is not in a position to determine 
whether its operations meet reasonable efficiency targets in 
comparison to the industry. It also has not assessed its own 
performance over time. 


¢ The Corporation does not have a human resource strategy for 
addressing its key challenges, including high levels of absenteeism 
and high employee turnover, which directly affect the day-to-day 
operations of the plant. 


In our opinion, given the pervasiveness and significance of the 
deficiencies mentioned, and based on the criteria established for 
the examination, the Corporation does not have the reasonable 
assurance required under section 131 of the Financial Administration 
Act that its assets are safeguarded and controlled, its resources are 
managed economically and efficiently, and its operations are carried 
out effectively. 


We noted that the Corporation has appropriate practices in place 
in the area of quality management, marketing and inventory 
management. 


The Corporation has responded. The Corporation agrees with our 
recommendations. 
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National Arts Centre Corporation—Main Points 


[We issued the full report to the board of directors on 19 July 2010. 
The examination work covered the period from September 2008 to 
December 2009. For the full report, please go to www.nac-cnda.ca or 
contact the Corporation. ] 


What we examined The National Arts Centre Corporation was established in 1966 under 
the National Arts Centre Act and began operating the National Arts 
Centre in 1969. The Corporation’s mandate is threefold: to operate and 
maintain the National Arts Centre, to develop the performing arts in the 
National Capital Region, and to assist the Canada Council for the Arts 
in the development of the performing arts elsewhere in the country. 


The Corporation reports to Parliament through the Minister of 
Canadian Heritage and Official Languages. It is governed by a Board of 
Trustees and, at the time of our examination, employed 900 people 
including 61 full-time musicians. Approximately 60 percent of the 
workforce is made up of part-time employees. In the 2008-09 fiscal 
year, the National Arts Centre had revenues of $30.8 million from box 
office sales, restaurants, parking, hall rentals, and the National Arts 
Centre Foundation, and parliamentary appropriations of $39.8 million 
for operating and capital expenses. The Foundation is a separate legal 
entity established in 2000 as a registered charity to raise funds from 
individuals, foundations, and corporations to support the National 
Arts Centre’s programs and other qualified donees whose objects are in 
line with the mandate of the Corporation. 


We examined whether the National Arts Centre Corporation’s systems 
and practices provide it with reasonable assurance that its assets are 
safeguarded and controlled, its resources are managed economically 
and efficiently, and its operations are carried out effectively. Since the 
National Arts Centre Foundation is a separate legal entity, it did not 
fall within the scope of our examination. The systems and practices we 
examined were those in effect from September 2008 to December 2009. 


Why it’s important == The National Arts Centre is the only bilingual, multi-disciplinary 
performing arts organization in North America, and is one of the 
largest in the world. Over its 40 year history, the Corporation has 
become one of Canada’s leading centres for live performance, the 
creation of new works, and arts education. The Corporation has 
six artistic departments: English theatre, French theatre, dance, 
music, variety and community programming, and the Scene festivals. 
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The Corporation fosters the next generation of artists through children 
and youth programming on its stages and by taking music, dance, and 
theatre into the school system. It also hosts the annual Summer Music 
Institute, a music education program that attracts some of the most 
talented emerging artists from around the world. The Corporation also 
produces teaching materials and study guides that are distributed to 
schools across the country by mail and through its website. 


What we found = Our examination found a significant deficiency in the Corporation’s 
systems and practices for the maintenance of the Centre. A significant 
deficiency is a major weakness that could prevent the Corporation 
from having reasonable assurance that its assets are safeguarded and 
controlled, its resources are managed economically and efficiently, and 
its operation are carried out effectively. 


¢ During our examination, the Corporation did not have effective 
maintenance management systems and practices to allow it to 
monitor the condition of the building and to ensure that necessary 
maintenance activities were carried out in a timely and efficient 
manner. 


We found sound systems and practices in a number of areas, 
particularly in activities that are core to the Corporation’s mandate. 
We also noted some areas that need improvement. 


¢ The Corporation has defined its strategic direction, and it relies 
on strong artistic leadership within each discipline to implement 
it. The Chief Executive Officer (CEO) ensures that this 
leadership is in place by seeking out artistic directors who are 
among the best in their field. Artistic directors make programming 
decisions based on their artistic vision, the Corporation’s mandate 
and strategic goals of fostering artistic excellence and education, 
and the needs of targeted audiences. The Corporation collects 
data through various sources to better understand its audiences 
and guide its promotional activities. 


¢ The Corporation has in place the key elements of a good 
governance framework, enabling its Board of Trustees to effectively 
carry out its oversight and stewardship functions. The CEO is 
selected and appointed by the Board, and he is clearly accountable 
to the Board. Oversight by the Board and committees is focused on 
those areas that most impact the Corporation’s performance. 
For example, in the past year, the Board regularly reviewed key 
corporate initiatives, such as the Corporation’s financial 
restructuring in response to the downturn in the economy. 
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* The Corporation relies on a strong financial management 
framework to deliver its artistic programs while respecting the 
budgeted resources. Artistic directors work with administrative 
directors to manage financial risks and costs within each 
discipline. The disciplines play a key role in monitoring revenues 
and costs within their approved budget. Artistic directors and 
administrative directors are directly accountable to the CEO. 


* The Corporation has established sound practices to manage the 
major rehabilitation project it initiated in 2006. The project 
management team actively monitors costs and scheduling and takes 
into account business continuity, health and safety, and financial 
risks to determine priorities. Despite the Corporation's sound 
project management practices, the major rehabilitation project is 
facing delays, changes in scope, and additional costs. These are 
largely due to unforeseen issues that could not be factored into the 
original planning for the major rehabilitation project. 


The Corporation has responded. The Corporation agrees with the 
recommendations. 
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Telefilm Canada—Main Points 


[We issued the full report to the board of directors on 10 June 2010. 

The examination work covered the period from May 2009 to January 2010. 
For the full report, please go to www.telefilm.gc.ca or contact the 
Corporation. ] 


Telefilm Canada is a Crown corporation established in 1967 under the 
Telefilm Canada Act. Its mandate is to foster and promote the 
development of the Canadian audiovisual industry. It supports the 
industry through investments, advances, and grants. Its resources are 
used to finance feature films, as well as multimedia and multiplatform 
projects. Telefilm Canada provides full support for the creation, 
production, marketing, and distribution of audiovisual works, with 
special attention to building the industry capacity. 


Telefilm Canada reports to Parliament through the Minister of Canadian 
Heritage and Official Languages. It is governed by a board of directors. 
The Corporation receives funding of approximately $109 million 
annually through a parliamentary appropriation. It provides financial 
and strategic support to the motion picture industry, and from this 
support stem investments from other financial sources. Telefilm also 
administers the programs of the Canada New Media Fund on behalf of 
the Department of Canadian Heritage, using an annual transfer 
payment of about $14 million provided by the Department for this 
purpose. It also administers a service agreement with the Canadian 
Television Fund valued in 2008-2009 at $287 million to fund television 
projects. Telefilm has about 200 employees. 


In conducting this first special examination of Telefilm Canada, we 
looked at whether the Corporation has systems and practices in place 
that provide it with reasonable assurance that its assets are safeguarded 
and controlled, its resources are managed economically and efficiently, 
and its operations are carried out effectively. Our audit work covered 
the systems and practices that were in place between May 2009 and 
January 2010. 


The Canadian audiovisual industry is culturally and economically 
important for Canada. Canadian film, television, and interactive 
digital productions have an approximate total value of $10 billion per 
year and provide work for some 178,600 people. Telefilm provides 
financial and strategic support for the creation and production of 
works and cultural products that reflect Canadian diversity, strengthen 
the Canadian identity, support Canada’s international presence, and 
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thereby contribute to Canada’s cultural and economic vitality. 
Canadian productions are sold in more than a hundred countries. 


What we found We did not identify any significant deficiencies in Telefilm Canada’s 
systems and practices. We noted good management practices and also 
a number of areas that could be improved. 


¢ The Corporation has sound governance practices. The Board of 
Directors has established a profile of the expertise and knowledge 
that its members should possess, and the roles and responsibilities 
of Board members are clearly defined. We noted that Telefilm has 
an appropriate strategic planning process that aligns the 
Corporation's programs and activities with its strategic objectives. 


¢ Telefilm’s management is aware of the major risks to which the 
Corporation is exposed. It carried out a review of current systems 
and implemented an action plan to make the necessary 
improvements. However, the Corporation could benefit from 
more highly integrated risk management, which would enable it to 
consolidate information about risks and to manage them more 
systematically and comprehensively. 


¢ The Corporation has established policies and procedures for the 
administration of its programs that generally provide for 
consistent and objective project management. However, the 
Sineweb management information system could be used more 
efficiently in some areas, which would optimize its use and 
improve operational efficiency. Furthermore, although Telefilm 
already has several different quality monitoring mechanisms in its 
project management process, some planned mechanisms were not 
fully implemented for feature films. 


* Telefilm does a good job of managing its human resources. It has 
succeeded in recruiting and retaining qualified people. However, 
the offer-of-employment letters that are used in place of 
employment contracts between employees and the Corporation 
do not always reflect current policies. Furthermore, the 
documentation of the performance evaluation process and the 
performance bonus management policy lacks consistency. 
Performance measurement ratings and the justification for 
bonuses paid were not always adequately documented. 


The Corporation has responded. The Corporation accepts all the 
recommendations. 
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SPECIAL EXAMINATIONS OF CROWN CORPORATIONS —2010 


About the Chapter 


Objective 


The objective of this chapter was to bring to the attention of Parliament the results of the special 
examination reports that the Office of the Auditor General transmitted to the boards of directors of 
Crown corporations between | January 2010 and 31 December 2010 and that the Crown corporations 
have subsequently made public. 


Scope and approach 


The chapter includes the main points of four special examinations where the Auditor General 
was the examiner. 


The approach to this chapter consisted of presenting information that was already public, providing 
information about what a special examination is, and providing the main points of the four special 
examinations. 

Audit team 


Assistant Auditor General: Clyde MacLellan 
Principal: Marise Bédard 
Director: Raymond Murphy 


Omnia Nassef 


For information, please contact Communications at 613-995-3708 or 1-888-761-5953 (toll-free). 
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